The IFAB, soccer's lawmaking body, wants to keep the use of video replays to an minimum if they are eventually introduced into the sport to help referees make match-changing decisions.
Technical director David Elleray added that International Football Association Board was also considering whether the crowd should be shown the replay while match officials were deliberating over an incident.
IFAB approved live testing of video assistant referees (VARs), who monitor the action on screens and call the match referee's attention to key mistakes or omissions, in March 2016.
FIFA President Gianni Infantino has already said that soccer's governing body would like to use video replays in the 2018 World Cup, and IFAB is expected to decide next March whether to authorize their use in the game permanently.
Elleray said replays should be used only for "clear errors in goals, penalties and direct red cards, plus mistaken identity."
"The idea is not to check every decision. ... It is to overturn the ones that make the headlines," he said, adding that he wanted "minimum interference, maximum benefit."
"We would rather have one review in four matches than four in every match," he said.
Replays for crowds
Asked whether it would be a good idea to show replays to the crowd, Elleray said: "We are discussing and considering at the moment. There are strong arguments for, strong arguments against."
Elleray said that in cases where play continued after a possible infringement, the referee should stop the game for a review "as soon as the ball is in a neutral part of the field."
However, he acknowledged that in rare cases it would be impossible to stop the game quickly; in such cases, he said, officials would simply allow play to go on and review the incident at the first opportunity.
"Ultimately, the main thing is getting it right," he said.
"It could one day happen that there is a possible penalty at one end but play goes straight down the other end and a goal is scored.
"In that case, depending on the outcome of the video, the goal would be disallowed and a penalty awarded to the other side."