如何阅读文献——文献阅读三步法paper-reading

August2,2013

S.Keshav

DavidR.CheritonSchoolofComputerScience,UniversityofWaterloo

Waterloo,ON,Canadakeshav@uwaterloo.ca

ABSTRACT

Researchersspendagreatdealoftimereadingresearchpa-pers.However,thisskillisrarelytaught,leadingtomuchwastedeffort.Thisarticleoutlinesapracticalandefficientthree-passmethodforreadingresearchpapers.Ialsode-scribehowtousethismethodtodoaliteraturesurvey.

1.INTRODUCTION

Researchersmustreadpapersforseveralreasons:tore-viewthemforaconferenceoraclass,tokeepcurrentintheirfield,orforaliteraturesurveyofanewfield.Atypi-calresearcherwilllikelyspendhundredsofhourseveryyearreadingpapers.

Learningtoefficientlyreadapaperisacriticalbutrarelytaughtskill.Beginninggraduatestudents,therefore,mustlearnontheirownusingtrialanderror.Studentswastemucheffortintheprocessandarefrequentlydriventofrus-tration.

FormanyyearsIhaveusedasimple‘three-pass’approachtopreventmefromdrowninginthedetailsofapaperbe-foregettingabird’s-eye-view.Itallowsmetoestimatetheamountoftimerequiredtoreviewasetofpapers.Moreover,IcanadjustthedepthofpaperevaluationdependingonmyneedsandhowmuchtimeIhave.Thispaperdescribestheapproachanditsuseindoingaliteraturesurvey.

2.THETHREE-PASSAPPROACH

Thekeyideaisthatyoushouldreadthepaperinuptothreepasses,insteadofstartingatthebeginningandplow-ingyourwaytotheend.Eachpassaccomplishesspecificgoalsandbuildsuponthepreviouspass:Thefirstpassgivesyouageneralideaaboutthepaper.Thesecondpassletsyougraspthepaper’scontent,butnotitsdetails.Thethirdpasshelpsyouunderstandthepaperindepth.

2.1Thefirstpass

Thefirstpassisaquickscantogetabird’s-eyeviewofthepaper.Youcanalsodecidewhetheryouneedtodoanymorepasses.Thispassshouldtakeaboutfivetotenminutesandconsistsofthefollowingsteps:

1.Carefullyreadthetitle,abstract,andintroduction2.Readthesectionandsub-sectionheadings,butignoreeverythingelse3.Glanceatthemathematicalcontent(ifany)todeter-minetheunderlyingtheoreticalfoundations

4.Readtheconclusions

5.Glanceoverthereferences,mentallytickingofftheonesyou’vealreadyreadAttheendofthefirstpass,youshouldbeabletoanswerthefiveCs:

1.Category:Whattypeofpaperisthis?Ameasure-mentpaper?Ananalysisofanexistingsystem?Adescriptionofaresearchprototype?2.Context:Whichotherpapersisitrelatedto?Whichtheoreticalbaseswereusedtoanalyzetheproblem?3.Correctness:Dotheassumptionsappeartobevalid?4.Contributions:Whatarethepaper’smaincontribu-tions?5.Clarity:Isthepaperwellwritten?

Usingthisinformation,youmaychoosenottoreadfur-ther(andnotprintitout,thussavingtrees).Thiscouldbebecausethepaperdoesn’tinterestyou,oryoudon’tknowenoughabouttheareatounderstandthepaper,orthattheauthorsmakeinvalidassumptions.Thefirstpassisade-quateforpapersthataren’tinyourresearcharea,butmaysomedayproverelevant.

Incidentally,whenyouwriteapaper,youcanexpectmostreviewers(andreaders)tomakeonlyonepassoverit.Takecaretochoosecoherentsectionandsub-sectiontitlesandtowriteconciseandcomprehensiveabstracts.Ifareviewercannotunderstandthegistafteronepass,thepaperwilllikelyberejected;ifareadercannotunderstandthehigh-lightsofthepaperafterfiveminutes,thepaperwilllikelyneverberead.Forthesereasons,a‘graphicalabstract’thatsummarizesapaperwithasinglewell-chosenfigureisanex-cellentideaandcanbeincreasinglyfoundinscientificjour-nals.

2.2Thesecondpass

Inthesecondpass,readthepaperwithgreatercare,butignoredetailssuchasproofs.Ithelpstojotdownthekeypoints,ortomakecommentsinthemargins,asyouread.DominikGrusemannfromUniAugsburgsuggeststhatyou“notedowntermsyoudidn’tunderstand,orquestionsyoumaywanttoasktheauthor.”Ifyouareactingasapaperreferee,thesecommentswillhelpyouwhenyouarewritingyourreview,andtobackupyourreviewduringtheprogramcommitteemeeting.

1.Lookcarefullyatthefigures,diagramsandotherillus-trationsinthepaper.Payspecialattentiontographs.Aretheaxesproperlylabeled?Areresultsshownwitherrorbars,sothatconclusionsarestatisticallysig-nificant?Commonmistakeslikethesewillseparaterushed,shoddyworkfromthetrulyexcellent.2.Remembertomarkrelevantunreadreferencesforfur-therreading(thisisagoodwaytolearnmoreaboutthebackgroundofthepaper).Thesecondpassshouldtakeuptoanhourforanexpe-riencedreader.Afterthispass,youshouldbeabletograspthecontentofthepaper.Youshouldbeabletosummarizethemainthrustofthepaper,withsupportingevidence,tosomeoneelse.Thislevelofdetailisappropriateforapaperinwhichyouareinterested,butdoesnotlieinyourresearchspeciality.

Sometimesyouwon’tunderstandapaperevenattheendofthesecondpass.Thismaybebecausethesubjectmatterisnewtoyou,withunfamiliarterminologyandacronyms.Ortheauthorsmayuseaprooforexperimentaltechniquethatyoudon’tunderstand,sothatthebulkofthepa-perisincomprehensible.Thepapermaybepoorlywrittenwithunsubstantiatedassertionsandnumerousforwardref-erences.Oritcouldjustbethatit’slateatnightandyou’retired.Youcannowchooseto:(a)setthepaperaside,hopingyoudon’tneedtounderstandthematerialtobesuccessfulinyourcareer,(b)returntothepaperlater,perhapsafterreadingbackgroundmaterialor(c)persevereandgoontothethirdpass.

2.3Thethirdpass

Tofullyunderstandapaper,particularlyifyouarere-viewer,requiresathirdpass.Thekeytothethirdpassistoattempttovirtuallyre-implementthepaper:thatis,makingthesameassumptionsastheauthors,re-createthework.Bycomparingthisre-creationwiththeactualpaper,youcaneasilyidentifynotonlyapaper’sinnovations,butalsoitshiddenfailingsandassumptions.

Thispassrequiresgreatattentiontodetail.Youshouldidentifyandchallengeeveryassumptionineverystatement.Moreover,youshouldthinkabouthowyouyourselfwouldpresentaparticularidea.Thiscomparisonoftheactualwiththevirtuallendsasharpinsightintotheproofandpresentationtechniquesinthepaperandyoucanverylikelyaddthistoyourrepertoireoftools.Duringthispass,youshouldalsojotdownideasforfuturework.

Thispasscantakemanyhoursforbeginnersandmorethananhourortwoevenforanexperiencedreader.Attheendofthispass,youshouldbeabletoreconstructtheentirestructureofthepaperfrommemory,aswellasbeabletoidentifyitsstrongandweakpoints.Inparticular,youshouldbeabletopinpointimplicitassumptions,missingcitationstorelevantwork,andpotentialissueswithexperimentaloranalyticaltechniques.

3.DOINGALITERATURESURVEY

Paperreadingskillsareputtothetestindoingaliteraturesurvey.Thiswillrequireyoutoreadtensofpapers,perhapsinanunfamiliarfield.Whatpapersshouldyouread?Hereishowyoucanusethethree-passapproachtohelp.

First,useanacademicsearchenginesuchasGoogleScholarorCiteSeerandsomewell-chosenkeywordstofindthreeto

fiverecenthighly-citedpapersinthearea.Doonepassoneachpapertogetasenseofthework,thenreadtheirre-latedworksections.Youwillfindathumbnailsummaryoftherecentwork,andperhaps,ifyouarelucky,apointertoarecentsurveypaper.Ifyoucanfindsuchasurvey,youaredone.Readthesurvey,congratulatingyourselfonyourgoodluck.

Otherwise,inthesecondstep,findsharedcitationsandrepeatedauthornamesinthebibliography.Thesearethekeypapersandresearchersinthatarea.Downloadthekeypapersandsetthemaside.Thengotothewebsitesofthekeyresearchersandseewherethey’vepublishedrecently.Thatwillhelpyouidentifythetopconferencesinthatfieldbecausethebestresearchersusuallypublishinthetopcon-ferences.

Thethirdstepistogotothewebsiteforthesetopcon-ferencesandlookthroughtheirrecentproceedings.Aquickscanwillusuallyidentifyrecenthigh-qualityrelatedwork.Thesepapers,alongwiththeonesyousetasideearlier,con-stitutethefirstversionofyoursurvey.Maketwopassesthroughthesepapers.Iftheyallciteakeypaperthatyoudidnotfindearlier,obtainandreadit,iteratingasneces-sary.

4.RELATEDWORK

Ifyouarereadingapapertodoareview,youshouldalsoreadTimothyRoscoe’spaperon“Writingreviewsforsys-temsconferences”[3].Ifyou’replanningtowriteatechnicalpaper,youshouldreferbothtoHenningSchulzrinne’scom-prehensivewebsite[4]andGeorgeWhitesides’sexcellentoverviewoftheprocess[5].Finally,SimonPeytonJoneshasawebsitethatcoverstheentirespectrumofresearchskills[2].

IainH.McLeanofPsychology,Inc.hasputtogetheradownloadable‘reviewmatrix’thatsimplifiespaperreview-ingusingthethree-passapproachforpapersinexperimen-talpsychology[1],whichcanprobablybeused,withminormodifications,forpapersinotherareas.

5.ACKNOWLEDGMENTS

Thefirstversionofthisdocumentwasdraftedbymystu-dents:HosseinFalaki,EarlOliver,andSumairUrRahman.Mythankstothem.IalsobenefitedfromChristopheDiot’sperceptivecommentsandNicoleKeshav’seagle-eyedcopy-editing.

Iwouldliketomakethisalivingdocument,updatingitasIreceivecomments.Pleasetakeamomenttoemailmeanycommentsorsuggestionsforimprovement.Thankstoencouragingfeedbackfrommanycorrespondentsovertheyears.

6.

[1]I.H.REFERENCES

McLean,“LiteratureReviewMatrix,”

http://psychologyinc.blogspot.com/[2]S.PeytonJones,“ResearchSkills,”

http://research.microsoft.com/en-us/um/people/simonpj/papers/giving-a-talk/giving-a-talk.htm

[3]T.Roscoe,“WritingReviewsforSystemsConferences,”

http://people.inf.ethz.ch/troscoe/pubs/review-writing.pdf[4]H.Schulzrinne,“WritingTechnicalArticles,”

http://www.cs.columbia.edu/∼hgs/etc/writing-style.html[5]G.M.Whitesides,“Whitesides’Group:WritingaPaper,”

http://www.ee.ucr.edu/∼rlake/Whitesideswritingrespaper.pdf