Performance appraisals are widely used in organizations and typically involve raters evaluating subordinates along a set of items designed to represent job performance. A defining aspect of appraisals is their cyclicality; they occur repeatedly over time, often annually. Unfortunately, the role of time in appraisal research has been generally overlooked, with most investigations using time-invariant definitions and operationalizations of key constructs. One such example is rater experience. Rater experience is generally considered a relevant variable because raters continuously acquire skills and knowledge whenever they provide ratings, and these acquisitions can affect their ratings. However, previous operationalizations of rater experience have neglected its temporal nature and have modelled it as a time-invariant (between-subjects) predictor of ratings. In the current research, I view appraisals as a cyclical occurrence within organizations and investigate how rater experience, defined and modelled as a time-variant predictor, affects ratings and how it affects the way raters conceptualize and understand job performance. To do so, I conducted two studies using appraisal data from a major South American company comprising information from several appraisal cycles. In Study 1, the effect of rater experience on ratings was analyzed with data from 9,233 employees and 893 raters in 29 different business units across five appraisal cycles (24,608 observations). Results indicated that rater experience and familiarity with ratees were associated with higher ratings and span of control was associated with lower ratings. I also found an interaction between rater experience and familiarity, such that the positive association between familiarity and ratings was stronger when raters were more experienced. In Study 2, I examined whether rater experience affected the conceptualization and measurement of performance. In the between-rater design, measurement invariance was analyzed using ratings from 514 raters with different experience levels within one appraisal cycle. In the within-rater design, ratings from 80 raters in their first three appraisal cycles were analyzed. I found no evidence for measurement non-invariance between raters with different experience levels. Results from the within-rater analysis suggested that the job performance factor structure was not equivalent across cycles. Implications for research and practice are discussed.