Standard Umpire Evaluation Sheets: Fill & Download for Free

Yes they do. MLB pays eight umpire supervisors and nine umpire observers to cover the 15 cities. Umpire supervisors contact the umpires when they visit while observers simply watch and report.Complex system in place to evaluate umpires"One is an evaluation report which covers how [umpires] moved around on the field," Marsh said. "How the crew worked together, how they react to the plays, how they handle situations, if they hustle, their demeanor on the field, their focus every pitch. There are about 12 different categories."The other report is an analysis report, which is any time there was any close call, it has to be reviewed by the supervisor or observer that is at the game to make sure the call is correct, and it is itemized on that sheet."MLB also uses the Supervisor Umpire Review and Evaluation (SURE) system to cover games where no one is physically at the game and track ball and strike calls, fair/foul calls and safe/out calls on the bases.In the early 2000s Questec was installed in parks and umpires graded on their ball/strike calls. The umpires filed a grievance claiming the Questec wasn’t accurate and resulted in unfair reports. That led to a change to the umpire’s union agreement allowing a review of video for those reports. In 2009 Questec was replaced by the Zone Evaluation (ZE) system.ZE grades calls on every pitch not put in play or fouled off based on video from high speed cameras."It will tell whether a pitch is 2.6-7 inches outside or 3.4 inches inside, or how many low or how many high," Marsh said. "It will tell them everything about that pitch, plus give them video of it, a graphic of it, where the pitch crossed the plate or didn't cross the plate."Umpires get their report card the day after the game and have the ability to review video for themselves is they wish.Umpire ball and strike calling increased steadily after the installation of ZE The chart below from Umpires Are Less Blind Than They Used To Be shows umpire accuracy since 2008. The post uses Lance Barksdale specifically which is why his results are highlighted.“The difference between Barksdale and a league average ump is about five correct calls per game; the difference between Barksdale and the league’s worst umpire is closer to 10 calls a game. On average, that’s about one judgment call per inning that a good ump is getting right and a bad ump is getting wrong. That might not sound like much, but if once every six outs a batter gets another swing after a third strike that wasn’t or a pitcher strikes a hitter out on a pitch that’s actually a ball, you can start to see the impact.”Umpires also receive a mid year and post season evaluation."Mid-year evaluations go over everything . . . Z-E scores, any plays that stood out -- any plays that they may have missed are itemized in that," Marsh said. "Any plays . . . they did not handle appropriately. There's a red mark and a 'does not meet' and a complete explanation of why they got it. All those things are on their mid-year evaluation for them to review. Every guy has to . . .read it and sign off on it. They go through the whole thing at the end of the season as well."The league doesn’t demote umpires as some suggest because of the difficulty in getting there in the first place. They do however monitor reports and retrain on an individual level where they see a tendency to consistently make errors."(an ump) missed six pitches out of 160-something pitches," Marsh continued, creating an example. "That's a great score. But all six of those pitches were, say, off the outside corner of a left-hand hitter, and I wanted to know, why were his only misses out there? So I'll . . .watch the scores, watch him a couple times. I'll pick something up; I'll guarantee you. I talked to numerous umpires this year just about that type of thing."Marsh is correct in most cases as Dusty Dellinger explains in the post on the Five Thirty Eight just linked.“It was amazing how my perspective of the strike zone changed when I got this technology,” Dellinger said. “I thought pitches were on the plate, until you get that data back. You see that some of those pitches were not on the plate. It wasn’t something that was done intentionally. It was just your perception of the strike zone. I was able to quickly make adjustments based on having that information, which was huge to me.”The improvement in calls since 2008 is largely on fastball strikes. Since fastballs are the pitch thrown the overwhelming majority of the improving that call will create a positive trend. The fact that nothing has changed in calling a pitch a ball is something to be addressedThe league watches for bad ejections, too and ejections are certainly down but much of that is due to the replay system making some plays an automatic ejection just like arguing balls and strikes.Umpires do get disciplined but those actions are usually kept under wraps; like any decent employer, MLB tries to preserve umpire privacy. Now and then however events are such that disciplinary actions are made public.In 2012 Bib Davidson was suspended one game for “repeated violations of the Office of the Commissioner's standards for situation handling.”In May 2013 a whole crew was disciplined after a blatant failure to follow the rules in a game between Houston and the Angels. MLB suspended Fieldin Culbreth for two games and fined him for "misapplication of Official Baseball Rule 3.05(b)/" The other three members of the umpiring crew also fined.In July 2013 Brian Runge was fired for a drug use violation.In 2014 Joe West was suspended one game after an altercation with Jonathan Papelbon. Papelbon received a seven game suspension.EDIT:It’s been pointed out that in 2017 MLB switched from Sportvision’s Pitch F/X system to their proprietary STARCAST system. STATCAST uses multiple high speed cameras to more accurately measure release point and velocity. How exactly STATCAST works relative to ZE I can’t say.STATCAST measures many things that have no effect on umpire ball and strike calling. ZE data is a subset of that data that is not publicly available. It’s probably true that ZE quality improved over time and that this improvement is reflected in umpire grading.MLB switch to Statcast causes speed bump on pitches | Toronto Star