Skip to main content
eScholarship
Open Access Publications from the University of California

Dermatology Online Journal

Dermatology Online Journal bannerUC Davis

The Effect Of Academic “Misrepresentation” on Residency Match Outcomes

Main Content

The Effect of Academic “Misrepresentation” on Residency Match Outcomes
Emanual Maverakis1,2, Chin-Shang Li3, Ali Alikhan1, Tzu-Chun Lin4, Nayla Idriss1, April W Armstrong1
Dermatology Online Journal 18 (1): 1

1. Department of Dermatology, University of California Davis, Sacramento, California
2. Department of Veteran Affairs Northern California Health Care System, Sacramento, California
3. Department of Public Health Sciences, University of California, Davis, California
4. Department of Statistics, University of California, Davis, California


Abstract

PURPOSE: Studies have uniformly demonstrated misrepresentation of accomplishments by applicants to residency programs; yet it is unknown whether such applicants have a competitive advantage in the MATCH. Herein we identify the factors that influence an applicant's successful selection to a competitive residency. METHOD: Comparative study of 317 dermatology residency applicants during the 2007 application season. Main outcome measure: successful matching of an applicant into a U.S. dermatology residency. RESULTS: Factors positively associated with matching in univariate analysis included age (p=0.023); membership, Alpha Omega Alpha honors society (p=0.007); medical school research rank (p=0.013); USMLE scores (p<0.001); and number of unpublished manuscripts (p<0.001). Factors not associated with matching included the number of published manuscripts (p=0.460) and the combined impact factor of all published manuscripts (p=0.490). Multivariate analysis revealed that the USMLE Step 1 score (p=0.001), medical school research rank (p=0.040), and total number of unpublished manuscripts (p=0.046) were significantly associated matching in dermatology. Male gender trended towards but did not reach significance (p=0.054). CONCLUSIONS: Applicants that list multiple unpublished manuscripts have a significant competitive advantage in matching into a dermatology residency, even if these manuscripts remain unpublished.



I. Introduction

As interest in internal medicine declines and the physician workforce marches towards greater specialization [1], competition for highly sought after residencies and fellowships has increased. With a limited number of open slots, not all U.S. seniors applying for these positions can be accommodated [2]. Thus, a detailed analysis of the factors influencing an applicant’s successful selection is of general interest to the medical community. Although rarely mentioned in their personal statements, factors such as controllable lifestyle, income, and work hours seem to motivate the ultimate career choice of medical students [3, 4]. These factors are at odds with those sought by the residency selection committees, which are charged with the duty of identifying qualified applicants and establishing a program’s “rank order” for the National Resident Matching Program (NRMP) [5]. Ranking is a difficult task as the factors associated with an applicant’s potential to develop into an outstanding physician are ill defined. Clearly, scores on the Medical College Admission Test (MCAT) have been associated with the likelihood of an applicant doing well on the United States Medical Licensing Examination (USMLE) [6, 7]. In turn, USMLE scores have been associated with the likelihood of an applicant doing well on National Board of Medical Examiners (NBME) subject and board certifying examinations [8-11]. Despite this, a positive correlation between exam scores and clinical performance based upon faculty evaluations is not as well documented [12-15]. Thus, one must evaluate all aspects of an applicant’s record when making recommendations for admission. Evidently, some programs are quite good at establishing a “rank order” of applicants for the NRMP match [16], but others report that their rank of an applicant has no correlation with the eventual performance of the applicant during residency [15, 17, 18]. There is currently no recipe for selecting a superior applicant, and, in retrospect, even the most competitive programs have matched suboptimal physicians. In order to refine the selection process, the first crucial step is to intricately determine the factors that are positively associated with an applicant’s likelihood of matching.

It is accepted that factors such as USMLE scores, research and AOA status influence an applicant’s competitiveness for a favorable rank; less known are the increased odds of matching bestowed by these factors. In addition, several factors such as the quality or “impact” of an applicant’s research have yet to be investigated. Furthermore, whereas the misrepresentation of academic accomplishments by applicants has been consistently documented across all residency and fellowship programs examined [19-34], it is unknown whether such misrepresentation impacts an applicant’s likelihood of matching. As competitive residencies are struggling to retain young academic faculty [35], the question remains: are these programs truly selecting the most highly-qualified physicians from their very competitive applicant pool? Utilizing the 2007 UC Davis dermatology residency applicant pool we sought to identify the factors that positively contribute to an applicant’s competitiveness and to determine if the most “worthy” applicants are being favored in the selection process.


II. Methods

Ethics Statement: This study was approved by the UC Davis Institutional Review Board (IRB) and was conducted according to the principles expressed in the Declaration of Helsinki. The IRB waived the requirements to obtain informed consent.

Study Population: We examined all dermatology residency applications to UC Davis for the 2007 match. A total of 317 applications were submitted to UC Davis through the Electronic Residency Application Service (ERAS).

Covariates and Outcome: We read each application individually and recorded the variables of interest. Specifically, we recorded the applicants’ gender, age, PhD status, Alpha Omega Alpha (AOA) status, foreign medical graduate status, the applicants’ medical school research and primary care ranking based on the 2007 US News and World Report, the 2007 National Institute of Health rank for academic institutions, USMLE Steps 1 and 2 scores, number of full-time dermatology research years, publications, scholarly work that had been submitted or in preparation, abstracts, and presentations at meetings.

For each applicant, we designated a “candidate impact factor” using the following formula: Σ= Number of publications in Journal i x Impact Factor of Journal i, where Journal i represents indexed journals. We used the 2007 journal impact factor published in the Thomson Reuter's Journal Citation Reports, which reflects the average number of citations during the two preceding years.

To determine whether a manuscript designated as “submitted” at the time of application was later published, we searched Medline and Embase databases to determine if these manuscripts were ever eventually published. ERAS applications to the UC Davis Department of Dermatology were required to be submitted by December of 2006 and our analysis was completed in June of 2009. Thus, manuscripts listed as submitted, in preparation, and in press had over 2.5 years to be published.

We randomly selected 17 manuscripts that were reported as “submitted to the Journal of the American Academy of Dermatology (JAAD)” by applicants. The managing staff of the JAAD searched through the submission database to verify whether these 17 manuscripts were indeed submitted to the journal prior to March 5, 2009.

The primary outcome of the study was a dichotomous variable of “match success,” defined by whether the applicant matched into an ACGME-accredited dermatology residency program. To ensure that we gathered the most accurate data on match outcomes, we ascertained match results from the NRMP, corroborated the NRMP information against a dermatology-specific database of matched candidates, and verified applicants’ match status information with individual U.S. residency program directors, coordinators, or other individuals with knowledge of the programs.


Statistical analysis

We performed univariate analyses of independent variables to investigate whether each independent variable was significantly associated with the outcome variable of match success by using simple logistic regression.

For the multivariable analysis, we applied the following criteria to select variables for entry: any variable whose univariate test had a p-value of log-likelihood ratio test less than 0.25 was a candidate for the multivariable logistic regression model. We used 0.25 as a screening criterion for variable selection based on the work by Mickey and Greenland in 1989 [36]. Furthermore, we introduced two additional variables into the multivariable model based on our assessment of their relevance: number of publications and the candidate impact factor at the time of application.

Finally, we incorporated these variables independently via the stepwise selection procedure with the criterion of significance level (<0.25) for entering independent variables and significance level (>0.3) for removing independent variables. We kept the variables “number of publications” and “candidate impact factor” in the multivariable logistic regression model. We performed all statistical analyses using SAS® statistical software, version 9.2 (SAS Inc., Cary, North Carolina). These analyses were performed using the two-tailed analyses, with significance level of 0.05.


III. Results

During the 2007 dermatology residency match, 317 applications were submitted to the UC Davis Department of Dermatology. Among these applicants, 211 women comprised 66.6 percent of the applicant pool, whereas the 106 men comprised the remaining 33.4 percent. Ninety-one applicants (31.8%) reported belonging to Alpha Omega Alpha society (AOA). Twelve applicants (3.8%) were foreign medical graduates. Twenty-six applicants (8.2%) did not have a dermatology residency program at their home institution. At the time of the 2007 match announcement, 172 applicants (54.3%) failed and 145 applicants (45.7%) succeeded in matching into a dermatology residency program. Additional demographic and academic factors from this group of applicants are summarized in Table 1.

Combined, the 317 applicants listed a total of 979 publications. Of these, 421 were published and 558 were listed as in preparation, submitted, or in press. Of the 558 unpublished manuscripts, 91 (16.3%) were eventually published. Among the 17 randomly chosen manuscripts that had been reported as “submitted to the JAAD,” only 6 manuscripts (35%) were indeed submitted to the JAAD according to the electronic submission tracking database.

According to the univariate analyses (Table 2), factors that were significantly associated with matching success were older age (OR 1.025, 95% CI 1.002-1.048), AOA membership (OR 2.001, 95% CI 1.205-3.222), better medical school research ranking (OR 0.98, 95% CI 0.965-0.996), better medical school NIH ranking (OR 0.992, 95% CI 0.984-0.999), higher USMLE step 1 score (OR 1.039, 95% CI 1.025-1.054), higher USMLE step 2 score (OR 1.031, 95% CI 1.013-1.050), greater number of published and unpublished manuscripts (OR 1.103, 95% CI 1.022-1.190), greater number of unpublished manuscripts (OR 1.225, 95% CI 1.087-1.380), greater number of first author unpublished manuscripts (OR 1.253, 95% CI 1.084-1.448), and a lower publication ratio (OR 0.448, 95% CI 0.218-0.921). The publication ratio is defined as manuscripts that were published subsequent to the application divided by the number of unpublished manuscripts at the time of application.

The final multivariate logistic regression model that resulted from the stepwise selection procedure included the following factors: number of publications, impact factor, female gender, age, medical school research ranking, USMLE Step 1 score, number of dermatology research years, and number of unpublished manuscripts. The multivariate regression analysis showed that higher USMLE Step 1 scores (OR 1.031, 95% CI 1.012-1.051, p=0.001), greater number of unpublished manuscripts at the time of application (OR 1.185, 95% CI 1.003-1.399, p=0.046), and better medical school research ranking (OR 0.98, 95% CI 0.962-0.999, p=0.040) significantly predicted match success, after adjusting for all other factors in the model (Table 3).


IV. Conclusions

Currently, the NRMP provides summary statistics of match outcomes. Although helpful, these data have a selection bias in that they include only applicants who officially enter the match. For competitive residencies, many applicants fail to receive an interview or for some other reason become discouraged in the application process and switch to a less competitive residency. Thus, it is impossible for the NRMP to report data for all applicants on an “intent-to-match” basis, similar to the way in which investigators may report results of a clinical trial on an “intent-to-treat” basis. In this regard, we have chosen to analyze data for all applicants who applied to the UC Davis dermatology residency program, regardless of whether or not they ultimately entered the dermatology match. One advantage of this approach is that it captures a greater number of unmatched applicants, which is helpful for identifying the factors associated with matching. It is, therefore, not surprising that our calculated match rate is only 45.7 percent, which is a little lower than the 54.7 percent reported by the NRMP [2]. Given that there were only 579 applicants to dermatology nationwide, our pool of 317 applicants is likely representative of the entire applicant pool. The lower match rate seen in our study is similar to the 49.7 percent success rate reported for otolaryngology when applicants were likewise analyzed on an intent-to-match basis [37].

Because of the limited numbers of MD/PhDs in our pool, we were unable to confirm the NRMP data supporting a higher match rate for these applicants. On the other hand, in accord with the NRMP, we report that AOA membership (p=0.007), medical school NIH research rank (p=0.034), and USMLE scores (p<0.001 for step 1 and step 2 respectively) are positively associated with an applicant’s likelihood of matching. We also found that the U.S. News and World Report medical school research rank was positively associated with the likelihood of an applicant matching (p=0.013). Multivariate logistic regression analysis also revealed that higher USMLE Step 1 scores (p=0.001) and better medical school research rank (p=0.040) significantly predicted match success, after adjusting for all other factors in the model.

One surprising feature of our data is that female applicants were less likely than their male counterparts to match in dermatology, a finding that trended towards, but did not reach, significance (p=0.079, univariate analysis; p=0.054, multivariate analysis). It is possible that the high ratio of female to male applicants (211 to 105) negatively impacted their likelihood of matching into dermatology. Alternatively, their lower USMLE scores could have been a factor. The mean USMLE step 1 score for our female applicants was 228.9 (SD=19.9) versus 235.6 (SD=18.65) for the male applicants. Gender difference in USMLE exam scores is a general finding not limited to dermatology applicants [38, 39]. Univariate analysis also revealed that older applicants had better odds of matching in dermatology (p=0.023). This might be correlated to their maturity or greater number of accomplishments.

Scientific misrepresentation is a well-recognized problem in medicine [40], and there have been reports of applicants misrepresenting their academic accomplishments when applying for admission to medical school, residency programs, fellowship programs, and even for faculty appointments [19-28, 30-34, 41, 42]. However, the ultimate question remains unanswered. Does misrepresentation bestow upon the applicant a competitive advantage? We sought to answer this question. Similar to the results reported by the NRMP, we demonstrate that research is positively associated with an applicant’s likelihood of successfully matching into a dermatology residency program. Yet, one striking finding from our analysis is that there was no association between the scientific “impact” of an applicant’s manuscripts and the likelihood of matching (p=0.490). The majority of applicants had no calculable impact (185/314) and almost half had no actual PubMed indexed publication (145/314). In contrast, there was a very strong association between the number of in preparation and submitted manuscripts and the likelihood of matching (p<0.001). Following this group longitudinally, we were able to re-evaluate these listed but not yet published manuscripts to determine how many of them actually made it to print. Results showed that a disappointing few (91 of 558) of the listed manuscripts were published. Even more striking, the ratio of eventually published to listed manuscripts was negatively associated with the likelihood of matching (p=0.026). Successful applicants listed up to 17 submitted and in preparation manuscripts, the majority of which were never published. Several applicants had a published-to-listed manuscript ratio of 0; i.e., none of their “manuscripts” made it to print. Given that the overall acceptance rate of these manuscripts was oddly low (16.3%), the possibility of fraud in the application process was investigated. Although the majority of the manuscripts listed as submitted did not indicate the actual journal to which they were submitted, some were listed as submitted to the JAAD. Detailed analysis of the JAAD submission records revealed that only six of the 17 manuscripts (35.3%) were actually submitted. Although these numbers are too few to extrapolate to the group as a whole, we are able to conclude that, at least on a limited scale, gross fraud existed within the application pool. Multivariate logistic regression analysis also demonstrated that a greater number of listed unpublished manuscripts was significantly associated with match success (p=0.046). Thus, listing unpublished manuscripts bestows a competitive advantage to the dermatology applicant, even if these never make it to print.

Although the overwhelming majority of studies have demonstrated misrepresentation of academic accomplishments by applicants, it is important to note that some groups have argued that misrepresentation is minimal if one expands their search to include multiple databases and exclude in press, submitted, and in preparation manuscripts from their study [22, 43]. Thus, the incidence of misrepresentation does vary from study to study. There are several reasons for this. First, there is no uniform consensus of what “misrepresentation” means. Some studies define misrepresentation as citing a nonexistent article from an actual journal or claiming authorship of an article that did not list the applicant as an author. Other studies have a more broad definition and acts of misrepresentation have included identifying a manuscript as published that was not yet accepted for publication. Making a change in the order of authors can also be considered an act of misrepresentation, an applicant might move himself or herself up to be more visible. As the detail of the analysis increases, the incidence of misrepresentation will also increase. Second, the types of publications evaluated will affect the incidence of misrepresentation. Some studies include only journal articles whereas others broaden their study to include abstracts, books, and book chapters. Third, the incidence might vary depending on the population examined. For example, some studies evaluated only those applicants who were actually interviewed whereas others include everyone who applied to their program. Our manuscript differs from most in that we conducted a longitudinal study that evaluated the likelihood of a listed manuscript actually making it to print. Although these were excluded from many studies, Boyd et al reported that only 30 percent of the in press manuscripts, 27 percent of the submitted manuscripts and 4 percent of the in preparation manuscripts were eventually published [22]. Because authorship and titles might change during the submission process, we allotted a considerable degree of freedom for this in our analysis. Basically, anything that was eventually published that resembled the listed manuscript was considered valid. However, it is important to note that there was a spectrum in the way applicants reported their publications. Some clearly reported that a publication was in a medical student journal or in an abstract booklet whereas others used ambiguous nomenclature that did not make this clear. Although some applicants might be inadvertently doing this, one potentially important quality of a good physician is the ability to convey things clearly in writing; some were clearly superior to others in this regard.

In summary, there is no correlation between the scientific impact of an applicant’s work and the likelihood of matching in dermatology. In contrast, the number of listed publications increases an applicant’s chances of matching in dermatology, regardless of the article’s likelihood of making it to print. Tragically, the applicants who accurately reported their manuscripts had a reduced chance of matching in dermatology, as the ratio of listed-to-published manuscripts was inversely related to an applicant’s chances of matching. Our results should have profound impact on how residency selection committees rank applicants. Clearly, there should be vetting processes during which the applicants provide proof of manuscript submission and copies of in preparation and submitted manuscripts, similar to the way in which official copies of transcripts are reviewed. Admissions committees would never rely upon self-reporting of grades or USMLE scores. Given the demonstrated importance of research in the admissions process, verification of this component of the application is essential. In the absence of such a verification protocol, individuals who accurately report their research accomplishments will continue to be at a disadvantage. Whereas inherent difficulties exist in evaluating an applicant’s professionalism, moral reasoning, and integrity, applicants who intentionally misrepresent their academic accomplishments may be inferior to their peers in terms of these less quantifiable but equally important attributes. Ideally, the NRMP will allow an applicant’s proof of manuscript submission, acceptance of publication, and published manuscripts to be uploaded as part of the application process. Until then, residency programs need to examine carefully the accuracy of application materials, critically evaluate applicants’ scientific contributions, and continually improve the selection process.

References

1. Hauer KE, Durning SJ, Kernan WN, Fagan MJ, Mintz M, O'Sullivan PS, Battistone M, DeFer T, Elnicki M, Harrell H, Reddy S, Boscardin CK, Schwartz MD. Factors associated with medical students' career choices regarding internal medicine. JAMA. 2008 Sep 10;300(10):1154-1164. [PubMed]

2. NRMP. Charting Outcomes in the Match: Characteristics of Applicants Who Matched to Their Preferrred Specialty in the 2007 NRMP Main Residency Match. National Resident Matching Program. 2007 August.

3. Dorsey ER, Jarjoura D, Rutecki GW. Influence of controllable lifestyle on recent trends in specialty choice by US medical students. JAMA. 2003 Sep 3;290(9):1173-1178. [PubMed]

4. Rosenthal MP, Diamond JJ, Rabinowitz HK, Bauer LC, Jones RL, Kearl GW, Kelly RB, Sheets KJ, Jaffe A, Jonas AP, et al. Influence of income, hours worked, and loan repayment on medical students' decision to pursue a primary care career. JAMA. 1994 Mar 23-30;271(12):914-917. [PubMed]

5. Roth AE. The origins, history, and design of the resident match. JAMA. 2003 Feb 19;289(7):909-912. [PubMed]

6. Donnon T, Paolucci EO, Violato C. The predictive validity of the MCAT for medical school performance and medical board licensing examinations: a meta-analysis of the published research. Acad Med. 2007 Jan;82(1):100-106. [PubMed]

7. Julian ER. Validity of the Medical College Admission Test for predicting medical school performance. Acad Med. 2005 Oct;80(10):910-917. [PubMed]

8. Ogunyemi D, De Taylor-Harris S. NBME Obstetrics and Gynecology clerkship final examination scores: predictive value of standardized tests and demographic factors. J Reprod Med. 2004 Dec;49(12):978-982. [PubMed]

9. Myles TD, Henderson RC. Medical licensure examination scores: relationship to obstetrics and gynecology examination scores. Obstet Gynecol. 2002 Nov;100(5 Pt 1):955-958. [PubMed]

10. Armstrong A, Dahl C, Haffner W. Predictors of performance on the National Board of Medical Examiners obstetrics and gynecology subject examination. Obstet Gynecol. 1998 Jun;91(6):1021-1022. [PubMed]

11. Shellito JL, Osland JS, Helmer SD, Chang FC. American Board of Surgery examinations: can we identify surgery residency applicants and residents who will pass the examinations on the first attempt? Am J Surg. 2009 Nov 4. [PubMed]

12. Boyse TD, Patterson SK, Cohan RH, Korobkin M, Fitzgerald JT, Oh MS, Gross BH, Quint DJ. Does medical school performance predict radiology resident performance? Acad Radiol. 2002 Apr;9(4):437-445. [PubMed]

13. Papp KK, Polk HC, Jr., Richardson JD. The relationship between criteria used to select residents and performance during residency. Am J Surg. 1997 Apr;173(4):326-329. [PubMed]

14. Davidson RC, Lewis EL. Affirmative action and other special consideration admissions at the University of California, Davis, School of Medicine. JAMA. 1997 Oct 8;278(14):1153-1158. [PubMed]

15. Borowitz SM, Saulsbury FT, Wilson WG. Information collected during the residency match process does not predict clinical performance. Arch Pediatr Adolesc Med. 2000 Mar;154(3):256-260. [PubMed]

16. Olawaiye A, Yeh J, Withiam-Leitch M. Resident selection process and prediction of clinical performance in an obstetrics and gynecology program. Teach Learn Med. 2006 Fall;18(4):310-315. [PubMed]

17. Thordarson DB, Ebramzadeh E, Sangiorgio SN, Schnall SB, Patzakis MJ. Resident selection: how we are doing and why? Clin Orthop Relat Res. 2007 Jun;459(255-259. [PubMed]

18. Adusumilli S, Cohan RH, Marshall KW, Fitzgerald JT, Oh MS, Gross BH, Ellis JH. How well does applicant rank order predict subsequent performance during radiology residency? Acad Radiol. 2000 Aug;7(8):635-640. [PubMed]

19. Sekas G, Hutson WR. Misrepresentation of academic accomplishments by applicants for gastroenterology fellowships. Ann Intern Med. 1995 Jul 1;123(1):38-41. [PubMed]

20. Baker DR, Jackson VP. Misrepresentation of publications by radiology residency applicants. Acad Radiol. 2000 Sep;7(9):727-729. [PubMed]

21. Bilge A, Shugerman RP, Robertson WO. Misrepresentation of authorship by applicants to pediatrics training programs. Acad Med. 1998 May;73(5):532-533. [PubMed]

22. Boyd AS, Hook M, King LE, Jr. An evaluation of the accuracy of residency applicants' curricula vitae: are the claims of publications erroneous? J Am Acad Dermatol. 1996 Oct;35(4):606-608. [PubMed]

23. Caplan JP, Borus JF, Chang G, Greenberg WE. Poor intentions or poor attention: misrepresentation by applicants to psychiatry residency. Acad Psychiatry. 2008 May-Jun;32(3):225-229. [PubMed]

24. Dale JA, Schmitt CM, Crosby LA. Misrepresentation of research criteria by orthopaedic residency applicants. J Bone Joint Surg Am. 1999 Dec;81(12):1679-1681. [PubMed]

25. Glazer JL, Hatzenbuehler JR, Dexter WW, Kuhn CB. Misrepresentation of research citations by applicants to a primary care sports medicine fellowship program in the United States. Clin J Sport Med. 2008 May;18(3):279-281. [PubMed]

26. Grover M, Dharamshi F, Goveia C. Deception by applicants to family practice residencies. Fam Med. 2001 Jun;33(6):441-446. [PubMed]

27. Gurudevan SV, Mower WR. Misrepresentation of research publications among emergency medicine residency applicants. Ann Emerg Med. 1996 Mar;27(3):327-330. [PubMed]

28. Gussman D, Blechman A. Verification of publications, presentations and posters by applicants to a residency in obstetrics and gynecology. J Reprod Med. 2007 Apr;52(4):259-261. [PubMed]

29. Konstantakos EK, Laughlin RT, Markert RJ, Crosby LA. Follow-up on misrepresentation of research activity by orthopaedic residency applicants: has anything changed? J Bone Joint Surg Am. 2007 Sep;89(9):2084-2088. [PubMed]

30. Kuo PC, Schroeder RA, Shah A, Shah J, Jacobs DO, Pietrobon R. “Ghost” publications among applicants to a general surgery residency program. J Am Coll Surg. 2008 Oct;207(4):485-489. [PubMed]

31. Panicek DM, Schwartz LH, Dershaw DD, Ercolani MC, Castellino RA. Misrepresentation of publications by applicants for radiology fellowships: is it a problem? AJR Am J Roentgenol. 1998 Mar;170(3):577-581. [PubMed]

32. Patel MV, Pradhan BB, Meals RA. Misrepresentation of research publications among orthopedic surgery fellowship applicants: a comparison with documented misrepresentations in other fields. Spine (Phila Pa 1986). 2003 Apr 1;28(7):632-636. [PubMed]

33. Roellig MS, Katz ED. Inaccuracies on applications for emergency medicine residency training. Acad Emerg Med. 2004 Sep;11(9):992-994. [PubMed]

34. Yang GY, Schoenwetter MF, Wagner TD, Donohue KA, Kuettel MR. Misrepresentation of publications among radiation oncology residency applicants. J Am Coll Radiol. 2006 Apr;3(4):259-264. [PubMed]

35. Loo DS, Liu CL, Geller AC, Gilchrest BA. Academic dermatology manpower: issues of recruitment and retention. Arch Dermatol. 2007 Mar;143(3):341-347. [PubMed]

36. Mickey RM, Greenland S. The impact of confounder selection criteria on effect estimation. Am J Epidemiol. 1989 Jan;129(1):125-137. [PubMed]

37. Calhoun KH, Hokanson JA, Bailey BJ. Predictors of residency performance: a follow-up study. Otolaryngol Head Neck Surg. 1997 Jun;116(6 Pt 1):647-651. [PubMed]

38. Weinberg E, Rooney JF. The academic performance of women students in medical school. J Med Educ. 1973 Mar;48(3):240-247. [PubMed]

39. Dawson B, Iwamoto CK, Ross LP, Nungester RJ, Swanson DB, Volle RL. Performance on the National Board of Medical Examiners. Part I Examination by men and women of different race and ethnicity. JAMA. 1994 Sep 7;272(9):674-679. [PubMed]

40. Engler RL, Covell JW, Friedman PJ, Kitcher PS, Peters RM. Misrepresentation and responsibility in medical research. N Engl J Med. 1987 Nov 26;317(22):1383-1389. [PubMed]

41. Goe LC, Herrera AM, Mower WR. Misrepresentation of research citations among medical school faculty applicants. Acad Med. 1998 Nov;73(11):1183-1186. [PubMed]

42. Wu JJ, Tyring SK. The integrity of dermatology applicants. Cutis. 2004 Jan;73(1):71. [PubMed]

43. Hebert RS, Smith CG, Wright SM. Minimal prevalence of authorship misrepresentation among internal medicine residency applicants: do previous estimates of “misrepresentation” represent insufficient case finding? Ann Intern Med. 2003 Mar 4;138(5):390-392. [PubMed]

© 2012 Dermatology Online Journal