SnapLogic Tutorials

Document Release: October 2013

SnapLogic, Inc. 2 West 5th Ave, Fourth Floor San Mateo, California 94402 U.S.A.

www.snaplogic.com

Table of Contents
SnapLogic Tutorials Table of Contents SnapLogic Overview Glossary The Tutorial Environment About the Tutorial Exercise 1: Working with Components Reviewing the Sample Leads to Prospects Pipeline Hands-on Exercise: Creating Components Creating a Writer Component Building the Read-Write Pipeline Exercise 2: Filtering Data Reviewing the California Prospects Pipeline Hands-on Exercise: Creating a Filter Component Duplicating Components Building the Read-Write-Filter Pipeline Exercise 3: Data Services Reviewing the Census Feed and Qualified California Pipelines Hands-On Exercise: Data Services Creating a Join Component Duplicating a Writer Component Building a Data Service Pipeline Exercise 4: Data Sorting Reviewing the Filtered Qualifying California Prospects Pipeline Hands-on Exercise: Sorting Data Exercise 5: Utilizing Pass-Through Reviewing the California Prospects Pipeline, Exercise 5 Hands-on Exercise: Implementing Pass-Through 1 2 4 4 4 5 6 6 8 11 12 16 16 17 19 20 22 22 25 27 28 29 32 32 34 38 38 39

-2-

SnapLogic Tutorials

-3-

1
SnapLogic Overview
SnapLogicistheonlyintegrationplatformbuiltonmodernwebstandardsand"containerized" Snaps,allowingyoutoeasilyconnectanycombinationofCloud,SaaSorOn-premiseapplicationsanddatasources.Likethewebitself,SnapLogicstandardizestheaccessprotocols, accessmethodsanddatastructureforeveryinteraction.TheseRESTful1interactionsallow datatobeexchangedbetweenserversinanorganizedwaythatmakesiteasyforthedatato beprocessedandinterpreted.Thiscreatesaresource-oriented,data-serviceslayerthatprovidestransformationandintegrationservicesforallkindsofintegrations.

Glossary
l Components:Componentsprovidebasicintegrationcapabilitiessuchasdatabasequeries,filereadandwrite,aggregate,sort,filter,join,andothers.Componentsareconfiguredwithspecificproperties. l Pipelines:Pipelinescanbeassembledintohierarchiesthatcanimplementcomplex logicandtransformationsondataforsophisticatedintegrations. l Repository:ResourcedefinitionsarestoredlocallyinaSnapLogicServersRepository.

The Tutorial Environment

ToillustratehowtobuildadataservicesinfrastructurelayerwithSnapLogic,thistutorial presentsfoursimpleexercises.Eachexercisebuildsuponthepreviousonetodemonstrate howexistingResourcescanbereusedaswellasintroducesnewSnapLogicComponentsand capabilities.Becauseexercisesbuildonthepreviousexercise,theyshouldbecompletedin orderastheydependonthesuccessfulcompletionofearlierwork. OnceyoucompletethistutorialyouwillknowhowtousetheSnapLogicDesignerinterfaceto: l ConfigureResourcesfordatasourcesandtargets l ConfiguretransformationResources l LinkComponentstoformPipelines l SpecifyfieldmappingsbetweenComponents l ExecutePipelinesfromtheDesigner l ReuseComponentsinotherPipelines ThistutorialisonlyanintroductiontothepowerofSnapLogic.TherearemoreComponents availablethatcanbereferencedintheDocumentationsectionlocatedat https://www.snaplogic.com/component-reference/component-reference.htm

-4-

SnapLogic Tutorials

tohelpyouunderstandhoweachComponentfunctions. Note: Thetutorialisbasedontheversionofthesoftwareatthetimeofthe releasedateonthefrontpage.Ifyouareusingadifferentversion,yourexperiencemayvary.

About the Tutorial

Thistutorialisbasedonahypotheticalexamplethatrequirestransformingagenericlistof salesleadsintoalistofqualifiedsalesprospects.Webeginbyprocessinganinputfileconsistingofleadsthatmightbegeneratedorpurchasedfromsomelistprovider(leads.csv). Thefileincludesfirstname,lastname,phone(home/work/cell)andaddress(street,city, state,andzip/postalcode). Theobjectiveistoidentifywhichoftheseleadsaresalesprospectsbyfilteringthembasedon locationandhouseholdincomelevel.Thisrequiresselectingonlythoseleadswithinaspecific territory(California)andmatchingthemagainsthouseholdincomedataforthelead'szip code.Householdincomeinformationisavailableatwww.irs.govandprovidesforeachzip codethepercentageofhouseholdsthathaveincomelessthan$10K,$10-$15K,$15-25K, $25K-50Kandgreaterthan$50K.Forthistutorial,dataforthestateofCaliforniaisaccessed byreadingalocalfile(CAIncomeByZip.csv). Theindividualtutorialsconsistof: l Exercise 1beginsbyreadingtheleads.csvfileandwritingitouttoafilewherethe outputfileomitsthehomeandcellphonenumbersfromtheleads.csvfile.Thisisdone bycreatingaPipelinewithtwoResources:AfileReaderandafileWriter. l Exercise 2usestheResourcesfromExercise1andcreatesanotherPipelinethatfilters theleadstoincludeonlytheleadsthatliveinCalifornia.TheWriterResourceisreused fromExercise1andaruntimeparameterissettouseanewfilenametopreservethe previousresults.(Parametersarevaluesthatcanbespecifiedatruntime,andaretypicallyusedforspecifyingthingslikefilenamesatruntime,soResourcesdon'thaveto beedited.) l Exercise 3readsthehouseholdincomedatafromthefileCAIncomeByZip.csvand joins(looksup)thatdatawiththefiltereddatafromExercise2.Itthenwritestoanew outputfile. l Exercise 4filtersandsortsthejoineddatabasedonzipcodeswherethepercentageof householdsthathaveanincomegreaterthan$50Kisaboveaspecifiedthreshold. l Exercise 5introducesyoutotheconceptofpass-through,wherefieldsnotneededfor aspecificComponentarepassedthroughforusedownstreaminthePipeline. Note: TheTutorialExercisePipelinesgeneratesampleoutputfileslikeex_n_prospects.csv,where`n`representstheexercisenumber(forexample,ex_1_prospects.csv isgeneratedinExercise1).Bydefault,theseoutputfileswillbe overwritteneachtimethePipelineisexecuted.

-5-

2
Exercise 1: Working with Components
Exercise1readsintheleads.csvfileandwritesoutafilewhichreorderstheFirstNameand LastNamefieldsandleavesoutthehomeandmobilephonenumbers. Toaccomplishthis,Exercise1constructsaPipelinewithtwoComponents:AfileReaderanda fileWriter.Theoutputfilewillbenamed:ex_1_prospects.csv. Note: TheTutorialPipelinesgeneratethesampleoutputfilesex_n_prospects.csv,(forexample,ex_1_prospects.csv,ex_2_prospects.csv).When yourunyourexercises,youwilloverwritethesefiles.Toavoidthis,youcan renameyourexerciseoutputfilestobeunique.Oryoucansimplyre-runthesamplepipelinestorecreatetheoutputfilesifyouneedtoexaminethem. YourversionofSnapLogicincludesanexampleforExercise1thatillustratesallthestepsyou needtotaketocreatethenecessaryPipeline. FirstletslookattheExercise1Components, thenwellcreateitourselves.

Reviewing the Sample Leads to Prospects Pipeline

AfterinstallingSnapLogicsuccessfully,connecttoSnapLogicDesignerusingthehostnameand portnumberselectedduringtheinstallation.Forexample: http://snaplogic1.snaplogic.com:8081 Double-clickontheEx1_Leads_to_ProspectsPipelinewithintheLibrary. PipelinesandComponentsareindicatedbydifferenticons,whichalsoindicatetheirstatus. Theseicons,aswellastheothericonsusedintheclient,aredescribedinthefollowingtable. Icon Type ThisisaservericonanditappearsontheServerList. ThisisafoldericonandisusedtoorganizePipelinesandComponentsonthe mainscreen. ThisisaComponenticon. ThisisaPipelineicon. ClickontheComponentnamedLeadstoviewitsconfiguration. l ThePropertiestabcontainsfiledelimiterandoutputfilevariablesforthisComponent. Note: Inthissampleexercise,thedelimiterandfilenamearespecifiedas$?{SOMENAME}.Thesyntax(dollarsign,questionmark,andbraces)indicatestotheComponentthatthevaluesforthesepropertiesareparametersthatwillbesetat runtime.Foranexampleofthisusage,seeHands-onExercise1:CreatingComponents.

-6-

SnapLogic Tutorials

l TheOutputtabdefinestheinput.csvfilefields,datatypes,anddescriptions. l TheParameterstabassignsdefaultvaluestothedelimiterandinputfilenamevariables. l ThePreviewtaballowsyoutoseethedataproducedbytheComponent.OnlyComponentswhichare"Producers",thosewhichhavenoinputviewsandonlyoutputviews, willhavethistabpresent. Clickon theComponentnamedProspectstoviewitsconfiguration. l ThePropertiestabdefinestheoutput .csvfiletobecreated(writtenout)byspecifyingifithasaheaderrow,whatthedelimiteris,andwhatthefilenameandpathis. ThistabalsoprovidesoptionaldescriptiveinformationforthisWriterComponent. l TheInputtabspecifiesthefields(name,datatype,anddescription)thataretoberead fromtheleads.csv file. l TheParameterstabassignsvaluestothedelimiterandinputfilenamevariables. ClickonthePipeline SettingsbuttonintheupperlefthandcornerofthePipeline.Clickthe followingtabs: l TheGeneral InfotabprovidesoptionaldescriptiveinformationforthisPipelineaswell asbuttonstoaccesstheRelatedPipelinesinformationandtheScheduler. l TheParameterstabshowswhatparametersareusedinthePipeline,andwhatthe defaultvaluesshouldbe.Anyparameterwhichdoesnothaveadefaultvalueisconsideredarequiredparameter,whichmustbeprovidedwhenexecutingthePipeline. ThistabpresentsonlytheinformationthataPipelineuserwouldneed. l TheRuntabcontainstheRunbuttonwhichstartsthePipelineprocessing.Thereisalso aparameterstablethatletsyouspecifyparametersatruntimethatmightbedifferent fromthedefaultvaluesindicatedontheParameterstab. Note: ThepurpleandgreenblocksontheLeadsandProspectsComponentsare output/inputviews.Inthiscase,thepurpleblockontheLeadsComponentisan outputviewforthedatathattheComponenthasreadin.Thegreenblockonthe ProspectsComponentisaninputviewforthisWriterComponenttoobtainthe datafromtheReader. ThelinkbetweenComponentsiscreatedbyclickingonapurpleoutputboxatthebottomof oneComponentanddraggingittoagreeninputboxatthetopofanother. ClickanywhereonthelinktodisplaytheFieldlinkingpage. TheFieldLinkerletsyoumapfieldsfromthesourcetothedestination.Eachinputfieldmust belinkedbeforethePipelinecanbeexecuted.Noteveryoutputfieldmustbeusedandyou canuseanoutputfieldmorethanonce.Here,theinputfortheWriterProspectsisbeing linkedtotheoutputfromtheReaderLeads.Therearetwowaystomaketheconnections.For eachinputfieldyoucanselectanoutputfieldfromthelistofvalues.YoucanalsoclickSuggesttohavefieldswiththesamenamesanddatatypesautomaticallylinked.

-7-

Exercise 1: Working with Components

WhenyouclicktheRunbutton,thePipelineprocessestheReaderoutput,mapsitintothe Writerinput,andcreatestheex_1_prospects.csvfile.Toconfirmthattheoperationwassuccessful,opentheex_1_prospects.csvfileandseethatthefieldsyoumappedarethefield containedinthefile.

Hands-on Exercise: Creating Components

Inthisexercise,youwillstarttorecreatetheLeadstoProspectsPipelinethatisusedthrough severaloftheexercises 1. Double-clickontheCSVReaderComponentintheFoundrypane. 2. IntheURIfieldenter:SnapLogic/User/Exercise_1/LeadswhereUseristheusernameyouwishtouse. Inthisexample,andforalloftheseexercises,wewillusethenameUser.Substitute yournameforUsereverywhereitisspecifiedintheseexercisesifdesired. Note: YoucanclickontheComponentnameintheGeneral tabtoaccess theonlinehelpforthatComponent. 3. OnthePropertiestab,leavethepropertiesunchangedexceptforthefollowing.(For detailsaboutthesepropertiesyoucanlookattheCSVReadComponentReferencedocumentation.) l Specifythedelimiterastheparameter$?{DELIMITER}.Wewillspecifytheactual delimitercharacteratruntime. l Specifytheskip_linesvalueas1.Thissettingallowsustoskipthespecified numberoflineswhenreadinganinputfilethatmightcontaincolumnheaders. Sinceleads.csvcontainsaheaderrow,wewanttoskipthisrow. l Specifytheinputfilenameas$?{INPUTFILE}.Wewillspecifytheactualnameof thefileatruntime. 4. ClickSave.Savingisoptionaluntilalltheeditsaremade. AllchangesarestoredintheclientbrowseruntiltheyaresavedtotheDataServer.If youhavemadechangestoaComponent,buthavenotsavedthem,youcandiscardthe changesbysimplyclosingthePipelinestab. TheLeadsComponenttabopens. Note: TheURIthatyouspecifiedhasgeneratedafolderhierarchyinthe DataServer.ForeachsubsequentComponent,besuretousethesamepath throughouttheseexercisessotheComponentsshowupinthesamefolders. Yourscreenshouldlooklikethefollowingfigure.

-8-

SnapLogic Tutorials

ToverifywhattypeofComponentyouareworkingwith,theComponenttypeisalways displayedintheGeneral InfotabandinthePropertiestabasshowninthepreceding figure. 5. SelecttheGeneral Infotab.ThissectioncontainsfieldsforsettingtheAuthorand DescriptionoftheComponent.Thesepropertiesareinformationalonlyandareoptional. YouwillnotneedtosetanyofthesepropertiesfortheTutorialexercises. 6. SelecttheParameterstab.YoushouldseethetwoparamatersDELIMITERandINPUTFILEadded.Ifnot, clickAdd Rowtwiceandnamethem. IfyouneedhelpconfiguringtheComponent,opentheversionoftheLeadsComponent inthe"Ex1_Leads_to_Prospects"Pipeline. 7. ForINPUTFILE, ,specifytheDefaultValuecolumnastheinputpathandfilename file://tutorial/data/leads.csv.Thispointstoafileinthe/snaplogic/data/tutorial/datadirectoryontheSnapLogicServer. 8. FortheDELIMITERparameter,enter,(comma)asthedefaultvalue. 9. ClickSave.Yourscreenshouldlooklikethefollowingfigure.

AtthispointwehavespecifiedenoughinformationtoallowtheSuggestfeatureto derivetheremaininginformation.IfaComponentsupportsauto-fill,thentheSuggest buttonwillbeenabledatthebottomoftheComponenttab.TheCSVReaderComponent can,givenafilename,derivethenumberofoutputfields.Itmayalsobeableto

-9-

Exercise 1: Working with Components

recommendthecorrectdelimiter,quotecharacter,andevenfieldnamesfortheoutput view. 10. ClickSuggest. Adialogopenswiththesuggestedoutputviewandasuggestedquotecharacter. 11. AcceptthesuggestionsbyclickingontheApply All Changesbutton. ClicktheOutputtab.YoushouldseetheviewOutput1withallthefieldnamescorrectly derived.IfyouchoosenottouseSuggest,youcancreatetheoutputviewmanuallyby clickingtheAdd OutputbuttonandclickFinishtoacceptthedefaultviewname(Output1). 12. TheoutputviewoftheLeadsReaderComponentshouldmatchthefieldsinthe leads.csvfile.ClickeachnameField001,Field002,andsoon,entermeaningfulnames, andspecifytheirdatatypeasstring.AllofthefieldsoftheLeadsComponentshouldbe definedasstrings. Keepinmindthatthefieldsinleads.csvcontainthefollowingtypesofinformation: 1. First Name 2. Last Name 3. Home Phone 4. Cell Phone 5. Work Phone 6. Street Address 7. City 8. State 9. Zip Code 13. Asampleoutputdefinitionsscreenisshowninthefollowingfigure.Notethatfield namescannotcontainspaces.Useunderscorecharactersifyouwanttoconcatenate wordssuchasNet ProfitbecomesNet_Profit.

- 10 -

SnapLogic Tutorials

14. ClickValidate.ValidationchecksthatyourComponentiswell-formedandperforms checkssuchasallrequiredpropertiesaresetandthenecessaryinput/outputviewsare defined. 15. ClickSave.TheReaderisnowdefined.

Creating a Writer Component

1. NavigatetotheSnapLogic > User > Exercise_1folderintheLibrarypane. Note: BynavigatingtotheintendedlocationforanewComponent,theNew ComponentdialogwillautomaticallyincludethispathinyourURI. 2. Double-clicktheCSVWriterComponentintheComponentLibrarypane. 3. IntheURIfield,addProspectstotheendsoitreads:SnapLogic/User/Exercise_ 1/Prospects. 4. ClickOK.TheProspectsWriterComponenticondisplays,andtheProspectsComponent opensforediting. 5. SelectthePropertiestab. 6. ForHeader Presentkeepthedefaultfalse (aswedonotneedaheaderrowinthe outputfile). 7. SpecifytheDelimiteras$?{DELIMITER}. 8. SpecifytheFilenameastheparameter $?{OUTPUTFILE}.Wewillspecifytheactual nameofthefileatruntime. 9. ClickSave. 10. SelecttheGeneral Infotab.Addyournameastheauthor,andprovideadescription. Thesefieldsareoptional.ClickSave. 11. SelecttheInputtabandclicktheAdd Inputbutton. a. AcceptthedefaultnameandclickNext. b. SelectCreate a new (blank) viewandclickFinish todisplaytheInput1view. NowweneedtotelltheWritertoonlywriteoutthelastname,firstname,street address,city,state,zipcode,andworkphonenumberfromtheleads.csvfile. 12. ClickAdd Rowandenterthefieldname,datatype,anddescriptionforField001aswe defineditintheReaderOutputfielddefinitions. 13. Repeatthisforlastname,address,city,state,zip,andworkphone. Hereareseveralthingsyoushouldnotice: l Inouroutputresult,wedonotwanthomephoneorcellphone,sothosefieldsare notaddedtotheinputviewdefinition.

- 11 -

Exercise 1: Working with Components

l Wehavespecifiedlastnamefirst.Theorderoftheinputfieldsdefinestheorder inwhichtheywillbewrittenout.Thisallowsustoreorderthefieldsfrom upstreamComponents. l SomeofthefieldnamesdonotmatchexactlywiththoseusedbytheLeadsComponent(forexample,Zipvs.Zip_Code).ThisisintentionalandwillbeusedtoillustratethatexactmatchesarenotnecessarywhenlinkingComponentstogether. Note: Everyfieldyoudefineinaninputviewhastobemappedfrom upstreamviews,sodonotdefinefieldsyoudonotwant.Also,forthese exercises,allfieldshavetobespecifiedusingstringasthedatatype. 14. ClickSave. 15. ClicktheParameterstabandclickAdd Row. 16. IntheNamecolumn,changePARAM_001tobetheoutputfileparameternameweintroducedinthePropertiestab,OUTPUTFILE. 17. IntheDefaultValuecolumn,specifytheoutputfilepathandname,file://tutorial/data/ex_1_prospects.csv. 18. ClickAdd RowandrepeatfortheDELIMITERparameter. 19. ClickSave. TheWriterComponentisnowdefined.

Building the Read-Write Pipeline

NextweneedtobuildaPipelinewiththeReaderandWriterComponentstoperformadata integration. 1. ClicktheNew Pipelinebutton. 2. IntheURIfield,addthenameLeads_to_ProspectstothepathSnapLogic/User/Exercise_1/. 3. ClickOK. 4. Clickonthecanvas. TheLeads_to_ProspectsPipelineiscreatedandisopenedforediting. 5. ClicktheGeneral Infotab. 6. Entera creatoranddescriptionforthisComponent.Thesevaluesareoptional. 7. ClickthePipelinetabwithinthePipelineProperties. 8. DragtheLeadsandProspectsiconsontothegridfield(alsoknownasthePipelinecanvas). Note:WhilenamingtheComponentsaswehavedoneisoptional,itisimportantto matchthenamesusedherewiththeonesusedintheParameterMap.

- 12 -

SnapLogic Tutorials

9. ClicktheLeadsoutputviewbox(purplebox)anddragtolinktotheProspectsinputview box(greenbox). Bydefault,theFieldLinkerdialogdisplaysautomaticallywhenever youconnecttwoComponents. 10. ClickApply. 11. Mapthefields.Foreachinputviewfield,selectoneoftheavailableoutputviewfields. YoucanalsoclicktheBbutton,whichwilllinkanyfieldswhosenamesanddatatypes match. Bydefault,SmartLinkwillbeinvokedautomaticallywhenyouconnecttwoComponents together.Again,thisbehaviorcanbecontrolledinthePipelineOptions.Assomefield namesdonotmatch,linktheremainingunlinkedfields. 12. ClickSave. 13. ClickPipeline Propertiesonthecanvastoolbar. 14. ClicktheParameterstab.WearenowgoingtodefinethePipelineparametersandhow theymaptothePipeline'sComponents. WhenyoudragaComponentontothePipelinecanvas,theComponent'sparametersare automaticallymappedasPipelineparameters.Double-clickonaparametertoeditit's nameorvalue.IfyoudonotwishtoexposeaComponent'sparameter,youcandeleteit byselectingtheparameterandclickingtheRemovebutton.Toaddaparameter,you canclicktheAddbuttontoenteranewparameter.Toaddaparameter,specifyaPipelineparameternameandselecttheComponentparametertowhichitmaps,anda defaultvalueifany. 15. Editeachparameterandchangethedefaultparameternames.BydefiningtheDELIMITERparametersasINPUT_DELIMITERandOUTPUT_DELIMITER,ratherthansimply DELIMITERaswasdefinedforbothComponents,wecannowusedifferentdelimitersif weneedto.Inthiscase,wewillleavebothdelimitersasacomma.Youcanalsousinga singlepipelineparameterandmapittomultipleComponentparameters.Forexample, ifyouknowthatallinputfilesusecommaasthedelimiter,youcouldmapasingle parameterFILE_DELIMITERtobothComponentDELIMITERparameters. 16. IntheDefault Valuefield,enterthevaluetobeusedwhenrunningthePipeline. 17. ClickSave. 18. ClicktheRuntab. TheRuntabcontainsthreesub-tabs,Run,PreviewDataandRuntimeInformation.The ParameterstableintheRunsub-tabispopulatedwiththeparameterstobeusedinthis Pipeline.Noticethatthedefaultvaluesappearinthetable. 19. Ifwewantedto,wecouldchangethevaluesrighthereontheRuntabandrunit. 20. ClicktheRunbutton. ThePipelinewillreadinthedataspecifiedintheReaderComponentandwriteittothe filespecifiedintheWriterComponent(ex_1_prospects.csv).ThestatusofthePipe-

- 13 -

Exercise 1: Working with Components

line'sexecutionisdisplayedintheRuntimeInformationsub-tab.Youcanaccess detailedinformationbyclickingontherowtoaccesstheShowLogmenu. 21. VerifythatthePipelineperformedthistaskbyopeningtheoutputfileandverifyingthat theFirst,Last,...Cityfieldsdisplayproperly. NoticethatwehaveachievedthegoalofcreatingaprospectsoutputfilefromtheoriginalleadsinputfilewheretheFirstNameandLastNamecolumnshavebeenreordered andwhichcontainsjusttheworkphonenumbersforalltheleads.Alsonoticethatthe inputfilecontains1,000records.Theoutputfilealsocontains1,000records. Note: Iftheex_1_prospects.csvfiledoesnotappearasexpected,openthe Show Logmenufortheruntoseeiftherewasanerror.

- 14 -

SnapLogic Tutorials

- 15 -

3
Exercise 2: Filtering Data
Exercise2duplicatestheComponentsfromExercise1andcreatesanotherPipelinethatfilters theleadstoincludeonlythoseinCalifornia.Toaccomplishthis,theFilterLeadsComponent examinestheStatecolumnintheleads.csvfileforthevalueequaltoCA.Thefilteredoutput iswrittentothefilenamedex_2_prospects.csv. Note: TheTutorialPipelinesgeneratethesampleoutputfilesex_n_prospects.csv,(forexample,ex_1_prospects.csv,ex_2_prospects.csv).When yourunyourexercises,youwilloverwritethesefiles.Toavoidthis,youcan renameyourexerciseoutputfilestobeunique.Oryoucansimplyre-runthesamplepipelinestorecreatetheoutputfilesifyouneedtoexaminethem. YourversionofSnapLogicincludesanexampleforExercise2thatillustratesthestepsto createthenecessaryComponentsandlinkthemwiththePipeline. ThisexercisewillshowyouhowtocreatethesetwoComponents,howtoduplicateandmodify theLeadsandProspectsComponentsfromExercise1,andcombinetheFilterLeadsComponentwiththeLeadsandProspectsComponentsinanewPipelinethatwillfilteroutallthe non-Californialeads. FirstletslookattheExercise2components,thenwellgocreateitourselves.

Reviewing the California Prospects Pipeline

Double-clickthepipelineEx2_CA_ProspectsanditwillopenupintheCanvas.TheComponentsusedinthisexercisewillbethesameonesusedforExercise1.Wewillreusethe LeadsComponentexactlyasitwasconfiguredinExercise1,butwewillspecifyaruntime parameterfortheProspectsComponentsothatitcanwritetoadifferentfile. ClickthePipeline PropertiesbuttonfortheEx2_CA_ProspectsPipelinetoseewhatthePipelineincludes.Clickthefollowingtabs: l TheGeneral tabprovidesdescriptiveinformationforthisComponent.Thisdatais optional. l TheInputtabdefinesthestructureofthedatathatisprovidedbytheupstreamComponentsintothePipeline.Inthisexercise,thePipelinegetsitsdatafromaCSVReader Componentthatreadsitfromalocalfile.ThatisdifferentfromdatathatentersaPipelinefromoneofitsinputviews.SincethereisnoupstreamComponentfeedingthePipeline,therearenoinputviews. l TheOutputtabdefinesdataexitingthePipeline.Inthisexercise,thePipelinehasa CSVWriterComponentthatwritesthedatatoalocalfile.Sincenodataisexitingthe Pipelinedirectly,ithasnooutputviews.

- 16 -

SnapLogic Tutorials

l TheParameterstabshowswhatparametersareusedinthePipeline,andwhatthe defaultvaluesareifany. l TheRuntabcontainstheRunbuttonwhichstartsthePipelineprocessing.Thereisalso aparameterstablethatletsyouspecifyparametersatruntimethatmightbedifferent fromthedefaultvaluesindicatedontheParameterstab. ThepurpleblockontheLeadsComponentisanoutputviewandthegreenblockontheFilterLeadsComponentisaninputview.ThelinkbetweenthemiscreatedbyclickingonaComponentthatcangenerateoutputanddraggingtoanotherComponent.Inthiscase,youclick LeadstodrawalinktoFilterLeads.YoucannotlinkProspectstoFilterLeadsbecausethe WriterComponentdoesnotprovideoutput. Nowdouble-clicktheFilterLeadsComponent: l ThePropertiestabdefinesthefieldtobeusedtofilterdata,itsvalue,andtheconditiontobemet.Inthisexample,tofindalltheCalifornialeads,weneedtospecifythat theColumnbeState,thevaluebeCA,andtheconditionbe==. l TheInputtabdefinesthestructureofthedatathatisprovidedbytheupstreamComponentsintotheFilter. l TheOutputtabdefinesthestructureofthedatafromtheFilterLeadsComponent.In thisexercise,Filteroutputviewshouldmatchitsinputview. ClickthelinkbetweentheLeadsComponentandtheFilterLeadsComponenttodisplaythe FieldLinker. ThefieldscominginfromtheReaderLeadsaremappedheretofieldsintheFilterLeadsinput viewbyclickingonthefieldsintheIncomingfields(availabletomap)panelanddragging themtotheFrompanel.ThenthesearematchedtotheFilterLeadsfieldsbydragging UnmappedoutgoingfieldstothecorrespondingTopanelsuchthatFromfieldsareonthe samelineasthefieldstherearegoingTo. ClickanywhereonthelinkbetweentheFilterLeadsComponenttotheProspectsComponentto displaytheLinkmenuandselectField LinkingtodisplaytheFieldLinkingpage. ThefieldscominginfromtheFilterLeadsaremappedheretofieldsintheWriterProspectsby clickingonthefieldsintheIncomingfields(availabletomap)panelanddraggingthemtothe Frompanel.ThenthesearematchedtotheWriterfieldsbydraggingUnmappedoutgoing fieldstothecorrespondingTopanelsuchthatFromfieldsareonthesamelineasthefields therearegoingTo. WhenyouclickRun,thePipelineprocessestheReaderoutput,passesitthroughtheFilterand intotheWriterinput,thencreatestheex_2_prospects.csvfile.Youshouldseethattheoutputresultcontains675records. NowletscreatetheComponentsinExercise2.

Hands-on Exercise: Creating a Filter Component

1. Double-clicktheFilterComponentintheFoundrypane. 2. SpecifytheURIasSnapLogic/User/Exercise_2/FilterLeads.

- 17 -

Exercise 2: Filtering Data

3. ClickOK. 4. SelecttheInputtab,clicktheAdd Inputbutton,andclickNextontheView Name pop-uptostarttheviewcreationwizard.Hereyouwillbeabletochoosebetweencreatinganewview,justlikewedidinExercise1,orcopyinganexistingviewfrom anotherComponent. 5. SelecttheCopy an Existing viewoptionandthenclickNext. 6. NavigatetotheExercise1LeadsComponentyoucreatedinthefirsttutorial.Whenyou selectaComponent,thewizardwilldisplayalistofanyinputoroutputviewsithas. Whenyouplaceyourmouseoveraview,youwillgetabriefsummaryofthefieldsin thatview.SelecttheoutputviewOutput1andclickFinish.Theoutputviewdefinition willbecopiedfromyourLeadsComponent. 7. WedonotrequirethePhone_handPhone_mfieldsinthisComponentsoclickthe"X" besidethoserowstodeletethefieldsfromthisview. 8. ClickSave. 9. ClicktheOutputtab. 10. RepeatthestepsyouperformedtodefinetheinputviewInput1tocreatetheoutput viewOutput1,butinsteadofcopyingtheviewfromtheLeadsComponent,justcopythe inputviewofyourFilterComponent. 11. ClickSave. 12. ClickthePropertiestab. 13. Specifythepropertiestobe: l Field:State l Operator:== l Value:CA l View:Output1 AnyComponentwhichdefinesapropertyasbeinganinputviewfieldnamewillpopulate thelistofvaluesforthatpropertywithvaluesderivedfromanyinputviewitfindsatthe time.Thelistofvalueswillbeemptyifnoinputviewhasbeendefinedyet.Youmust definetheinputviewfirstbeforeattemptingtoenteravaluefortheproperty.This restrictionappliestopropertieswhichreferenceinputoroutputviewnamesorfield names. 14. ClickClose. ThiscreatesaFilterLeadsComponentthatwillsearchtheStatecolumnoftheincoming datastreamandselectallrecordsthathaveCAasthestatename. 15. ClicktheGeneral Infotabandenteranauthorname,anddescriptionthenclickSave. Thesevaluesareoptional.

- 18 -

SnapLogic Tutorials

16. SkiptheParameterstab.SincewehavenotdefinedanyparametersforthisComponent,donotmakeanychangestothistab. TheFilterisnowdefined.

Duplicating Components
Duplicating a Reader Component
TheLeadsComponentcreatedinyourExercise1folderisthesameaswhatweneedfora Readerinthisexercise.SoletsduplicatethatversionoftheReaderanduseithere. ThisstepdivergesfromthesampleTutorialversionofExercise2inthatthereisnoLeads Componentduplicatedinthesampleversion.InthesampleversionofExercise2,yousimply dragtheLeadsiconfromtheExercise1folderanddropitontheExercise2Pipelinecanvas. Bothmethodsarevalid.However,theinstructionsinthisHands-onsectionassumethatyou duplicatetheLeadsComponent.DuplicatingaComponentallowsyoutomodifyaComponent withoutaffectingthepipelinesthatreferencetheoriginalComponent. YoucanalsoCTRL+Dragitemswithinthetreetocreateduplicates,orusethe"Copy"commandfromthe"Edit"menu. 1. Double-clicktheLeadsComponenticonfromTutorialExercise1toopenit. 2. SelectSave AsfromtheSavemenubutton. 3. FromtheSaveComponentAs...pop-up,specifytheURISnapLogic/User/Exercise_ 2/Leads. 4. ClickOK. TheduplicatedLeadsComponentappearsintheLibrary. SincetheLeadsReaderwillfunctionexactlyasitdidinExercise1(thatis,readinginthe fieldsfromtheleads.csvfile)wecanleaveitsProperties,BasicProperties,Output,and Parametersunchanged.

Duplicating a Writer Component

TheProspectsComponentcreatedinyourExercise1foldercanbeusedastheWriterComponentinthisexercise.SoletscopythatWriterComponent. 1. ClicktheProspectsComponenticonfromTutorialExercise1. 2. SelectSave AsfromtheSavemenubutton. 3. FromtheSaveAs...pop-up,specifytheURISnapLogic/User/Exercise_2/Prospects. 4. ClickOK. SincetheProspectsWriterwillfunctionasitdidinExercise1,andbecausewehaveused parametersfortheoutputfile(whichwecanchangeatruntimefrom ex_1_prospects.csvto ex_2_prospects.csv)wecanleaveitsProperties,BasicProperties,Output,andParameters unchanged.

- 19 -

Exercise 2: Filtering Data

Building the Read-Write-Filter Pipeline

NextweneedtolinktheLeads,Prospects,andFilterLeadsComponentsinanewPipelineto processthedata. 1. ClicktheNew Pipelinebutton. 2. SpecifytheURIasSnapLogic/User/Exercise_2/CA_Prospects. 3. ClickOK. 4. DragtheFilterLeadsComponentontothePipelinecanvas. 5. DragyourLeadsandProspectsComponentsontothePipelinecanvas. 6. LinktheLeadsReadertotheFilterLeadsComponent. Bydefault,theFieldLinkerdialogappears,andallofthefieldswillbemappedautomaticallybecauseamatchingfieldnamewasfoundforeachTofield. 7. ClickOKwhenfinished. 8. LinktheFilterLeadsComponenttotheProspectsWriter. 9. WhentheLinkdialogappears,someoftheTofieldswillremainunmappedbecausethe Fromfieldnamesdonotmatchexactly.MaptheFromfieldstotheremainingunmapped Tofields. 10. ClickOKwhenfinished. 11. FromthePipeline,clicktheParameterstab. 12. Edittheparametersandchangethedefaultnamesandvaluesasfollows.Specifythe Readerinputfile,theWriteroutputfile,andthedelimiters(commas)forbothComponents. Pipeline Parameter PARAM_000 PARAM_001 PARAM_002 PARAM_003 13. ClickSave. 14. ClicktheRuntab. ThePipelineParameterstable isnowpopulatedwiththeparameterstobeusedinthis Pipeline.Noticethatthedefaultvaluesappearinthetable.Ifwewantedto,wecould changethevaluesrighthereontheRuntabandrunit. 15. ClickRun Pipeline Now. ThePipelinewillreadinthedataspecifiedintheLeadsReaderComponent,applythefilteringcriteriatopassontherecordswithCAastheStatevalueandwritethemtothe filespecifiedintheProspectsWriterComponent(ex_2_prospects.csv). Mapped to Leads_1.DELIMITER Leads_1.INPUTFILE Prospects_1.outPUTFILE Prospects_1.DELIMITER , file://tutorial/data/leads.csv file://tutorial/data/ex_2_prospects.csv , Default Value

- 20 -

SnapLogic Tutorials

16. Openex_2_prospects.csvtoverifythefilteredresults.Youshouldseeonlythe675 leadsthathaveCaliforniaaddresses.

- 21 -

4
Exercise 3: Data Services
Exercise3startswithalloftheCalifornia-basedleadsandaddsinformationaboutincomelevelsbyzipcode.ThisexerciseintroducestheconceptofaDataServicePipeline.ADataServicePipelineisonethatprovidesafeedofdatathatcanbeusedbyotherPipelinesorother applications.DatafromthesepipelinescanbepresentedindifferentRepresentationswhen usedbyotherapplications.WewillcreateasimpleDataServicePipelinethatprovidesafeed ofincomedatatakenfromthefileCAIncomeByZip.csv.Next,wewillintroduceaJoinComponentthatreadstheincomedatafromtheDataServicefeedandjoinsit(looksup)withthe filtereddatafromtheleadsLeadsandFilterLeadsComponents.Itthenwritestheoutputtoa filenamedex_3_prospects.csv.Thisfilestillincludesalloftherecordsfromex_2_prospects.csv,butthistime,incomeinformationisadded. Note: TheTutorialPipelinesgeneratethesampleoutputfilesex_n_prospects.csv,(forexample,ex_1_prospects.csv,ex_2_prospects.csv).When yourunyourexercises,youwilloverwritethesefiles.Toavoidthis,youcan renameyourexerciseoutputfilestobeunique.Oryoucansimplyre-runthesamplepipelinestorecreatetheoutputfilesifyouneedtoexaminethem. YourversionofSnapLogicincludesanexampleforExercise3thatillustratesallthestepsyou needtotaketocreatethenecessaryComponentsandlinkthemwiththePipeline. FirstletslookattheExercise3components,thenwellgocreatethemourselves.

Reviewing the Census Feed and Qualified California Pipelines

FromtheDataServericon,youshouldseetheEx3_CensusFeedandEx3_Qual_CA_Prospects Pipelines. Double-clickCensusandthefollowingtabsareavailable: l ThePropertiestabdefinesthefiledelimiterandoutputfilename.Italsoprovides descriptiveinformationforthisReaderComponent.Thisdataisoptional. l TheOutputViewstabdefinestheCSVfilefields,datatypes,anddescriptions. l TheParameterstabdefinesthevaluesofthefiledelimiterandoutputfilename. Double-clickProspectsandthefollowingtabsareavailable: l ThePropertiestabdefinestheoutputCSVfiletobecreated(writtenout)byspecifying ifithasaheaderrow,whatthedelimiteris,andwhatthefilenameandpathis.Thistab alsoprovidesdescriptiveinformationforthisWriterComponent.Thisdataisoptional. Thedelimiterandfilenamearespecifiedas$?{SOMENAME}.Thedollarsign,question

- 22 -

SnapLogic Tutorials

mark,andbracesindicatetotheComponentthatthevaluesforthesepropertiesare parametersthatwillbesetatruntime(thatis,whenthePipelineprocessesassociated Components). l TheInputtabspecifiesthefields(name,datatype,anddescription)thataretobe writtentotheoutputfile.ThismatchesthefieldsdefinedintheReaderOutputtab. l TheParameterstabdefinesthevaluesofthefiledelimiterandoutputfilename. Double-clickJoinandthenclick: l ThePropertiestabdefinesthejoinexpressionsandthefieldaliasingproperties.It alsoprovidesoptionaldescriptiveinformationforthisWriterComponent. TheJoinExpressionsdialogcontainsthelistofexpressionsusedtojointheinputviews. Ifthereismorethanonesuchexpression,thereisanimplicitANDbetweenthem.A joinexpressiontakesthefollowinglogicalform: (Input1.name == Input2.name). TheAliasesdialogisusedtodisambiguatetheoutputviewfieldnames.Sincetheinput viewsthatarebeingjoinedoftenhavefieldsofthesamename,inordertodistinguish thesefields,anyambiguousoutputfieldsmustbealiased.Afully-qualifiedinputfieldis givenanalias,whichishowthisfieldwillberepresentedintheoutput.Forexample, consideraJoinComponentwith l twoinputviews:Input1:name,zipandInput2:name,company.Thesewillbe joinedbyname,(Input1.name=Input2.name) Andanoutput:name,company,zip. Joinneedstoknowwhichofthesenamesshouldbeusedfortheoutput.ThatiswhatAliasesspecify. l TheInputtabsspecifythefields(name,datatype,anddescription)thataretoberead fromthe leads.csvandCAIncomeByZip.csvfile. l TheOutputtabspecifiesthefields(name,datatype,anddescription)thataretobeoutputfromtheJoinComponent. l TheParameterstabdefinesanyparametersusedbytheComponent.Inthisexample theJoinComponentdoesnotuseanyparameters. Double-clicktheCensusFeedPipelinetoseewhatthePipelineincludes.Clickthefollowing tabs: l TheGeneral InfotabprovidesdescriptiveinformationforthePipeline.Thisdatais optional. l TheInputtabdefinesthestructureofthedatathatupstreamComponentsprovideto thePipeline.Inthisexercise,thePipelinegetsitsdatafromaCSVReaderComponent thatreadsitfromalocalfile.ThatisdifferentfromdatathatentersaPipelinefromone ofitsinputviews.SincethereisnoupstreamComponentfeedingthisPipeline,thereare noinputviews.

- 23 -

Exercise 3: Data Services

l TheOutputtabdefinesdataexitingthePipeline.TheCensusFeedPipelineisdirecting theoutputfromtheCensusComponent'soutputviewOutput001tothePipeline'soutput viewOutput1. l TheParameterstaballowsyoutodefinethePipelineparametersandmaptheparameterstothePipeline'sComponents. l ThePipelinetabshowsthe singleComponent. l TheRuntabcontainstheRunbuttonwhichstartsthePipelineprocessing.Thereisalso aparameterstablethatletsyouspecifyparametersatruntimethatmightbedifferent fromthedefaultvaluesindicatedontheParameterstab. Double-clicktheQual_CA_ProspectsPipelinetoseewhatthePipelineincludes.Clickthefollowingtabs: l TheGeneral InfotabprovidesdescriptiveinformationforthePipeline.Thisdatais optional. l TheInputtab,liketheCensusFeedPipelineabove,definesnoinputviews. l TheOutputtabdefinesdataexitingthePipeline.Inthisexercise,thisPipelinehasa CSVWriterComponentthatwritesthedatatoalocalfile.Sincenodataisexitingthe Pipelinedirectly,ithasnooutputviews. l TheParameterstaballowsyoutodefinethePipelineparametersandmaptheparameterstothePipeline'sComponents. l ThePipelinetabshowshowtheComponentsarelinked. l TheRuntabcontainstheRunbuttonwhichstartsthePipelineprocessing.Thereisalso aparameterstablethatletsyouspecifyparametersatruntimethatmightbedifferent fromthedefaultvaluesindicatedontheParameterstab. ClickthelinkfromLeadstoFilterLeadstodisplaytheLinkmenuandselectFieldLinkingtodisplaytheFieldLinkingpage. ThefieldscominginfromtheLeadsaremappedheretofieldsintheFilterLeadsbyclickingon thenamesintheIncomingfields(availabletomap)panelanddraggingthemtotheFrom panel.ThenthesearematchedtotheFilterLeadsfieldsbydraggingUnmappedoutgoingfields tothecorrespondingTopanelsuchthatFromfieldsareonthesamelineasthecorresponding Tofieldnames. ClickthelinkfromCensusFeedtoJoinandselectFieldLinkingtodisplaytheFieldLinking page. ThefieldscominginfromCensusFeed(theDataServicePipeline)aremappedheretofieldsin Join. WhenyouclickthelinkfromFilterLeads(theFilterComponent)toJoin(theJoinComponent) andselectFieldLinkingtodisplaytheFieldLinkingpage,youseehowthefilteredfieldsfrom FilterLeadsaremappedtotheJoinComponentwiththefieldsinCensus. TounderstandhowJoincombinestheFilterLeadsandCensusfields,leftclickthelinkfrom JointoProspects.SelectField LinkingtodisplaytheFieldLinkingpage.

- 24 -

SnapLogic Tutorials

Noticethatforthisexercise,theinputviewoftheProspectsComponenthasbeenchangedto includethenewfield:MoreThan50K. WhenyouclickRun,thePipelineprocessesthedatafromtheLeadsandCensusComponents, andsendsthedatatotheProspectsComponent,whichcreatestheex_3_prospects.csvfile. NowletscreatetheComponentsinExercise3.

Hands-On Exercise: Data Services

ThisexerciseusestwoReaderComponentstoreaddatafromtwodifferentsourcefiles.We reusethefiltereddatafromExercise2,createanewReaderComponenttoreadindatafrom anothersourcefile,thenjointhetwodatasourcestowriteoutthedesiredfile.TheWriter ComponentinthisexerciseisverysimilartotheWriterComponentwecreatedinExercise2, sowecanduplicateittosavetime.

Creating a Data Service Component

1. Double-clicktheCSVReaderComponentintheComponentLibrarypane. 2. NametheURISnapLogic/User/Exercise_3/Census. 3. ClickOK. 4. ClickthePropertiestab. 5. SettheDelimiterto$?{DELIMITER}. 6. SettheFile nameto$?{INPUTFILE}. Ratherthanexplicitlydefiningthedelimiterandtheinputfilename,weuseparameters thatcanbedefinedatruntime. 7. SetSkip linesto1.Thedatafilecontainsaheaderrowthatwewanttoskip.Leavethe otherpropertysettingsunchanged. 8. ClickSave. 9. ClicktheParameterstabandconfirmthattheDELIMITERandINPUTFILEparameters arelisted. 10. ClickSave. 11. SelecttheOutputtab,clicktheAdd Output. 12. Acceptthedefaultnameandcreatetheviewbasedonablankview. TheoutputviewfortheCensusReaderComponentrequiresthatyoudefinethefivecolumnsinCAIncomebyZip.csvandassignmeaningfulnames.Thefieldsare: l Zip l LessThan10K l MoreThan10K l MoreThan25K l MoreThan50K

- 25 -

Exercise 3: Data Services

13. ClickAdd Rowfivetimesandassignfieldnamesanddatatypesasfollows. Field Name Zip LessThan10K MoreThan10K MoreThan25K MoreThan50K Type string number number number number ZipCode Incomelessthan$10Kperyear Incomebetween$10Kand$25K Incomebetween$25Kand$50K Incomegreaterthan$50K Description

Forthepurposesofthisexample,thedatatypeforZipmustbestringforthisexercise toworkcorrectly.Theremainingfieldsmustbenumbertypes. 14. ClickSave. TheCensusReaderisnowdefined.NextwewillcreatetheData ServicePipelineincorporatingtheCensusComponent. 15. ClicktheNew Pipelinebutton. 16. IntheURIfield,enterSnapLogic/User/Exercise_3/CensusFeed. 17. ClickOK. TheCensusFeedPipelinetabshouldopen. 18. DragtheCensusComponentontothePipelinecanvas,namingitCensus. 19. ClickSave. 20. ClickPipeline Properties. 21. IntheOutputtab,clicktheAddbutton. 22. NamethePipelineoutputviewOutput001andmapittoCensus.Output1,thenclickFinish. 23. ClickSave. 24. ClicktheParameterstabandedittheparameterdefinitionstomatchthefollowing: l Pipeline Parameter:INPUT_DELIMITER l Mapped To:Census.DELIMITER l Default Value:, l Pipeline Parameter:CENSUS l Mapped To:Census.INPUTFILE l Default Value:file://tutorial/data/CAIncomeByZip.csv 25. ClickSave. 26. ClicktheRuntab.IntheRunsub-tab,youcanseethatthePipelinehasanoutputview availabletoreadfrom.

- 26 -

SnapLogic Tutorials

27. ClickthePreviewbutton.ThiswillrunthepipelineandsendthePipeline'soutputtothe Previewsub-tab. 28. ClickthePreview Datasub-tabtoseethedataoutputbythedataservice. AnotherwaytodemonstratetheutilityoftheDataServicePipelineistoviewitdirectlyfrom anotherbrowserwindow.OpenanewbrowserwindowandentertheURIasfollows: http://servername:8088/feed/SnapLogic/User/Exercise_3/CensusFeed/Output001?sn.content_type=text/html where servername:8088isreplacedwiththenameandportofyourSnapLogicDataServer. The/feed prefixisnecessarytoindicatetotheserverthataGETtothisURIwilltriggerPipelineexecution.Theviewnamesuffixishowyouselectwhichoutputisbeingrequested.The browser'sHTTPrequestwillstarttheCensusFeedPipelineandreturntheoutputviewdata fromOutput001inasimpleHTMLrepresentation.Formoreinformationonavailablerepresentations,seethesectionentitled"OutputDataRepresentationFormats"intheSnapLogic UserGuide. YoucanalsoswitchtotheGeneral tabandclickontheComponenthyperlinktoaccessalist ofavailablefeeds.Foryourconvenience,theDesignerwilllaunchtheseinanewwindow whenclicked.

Creating a Join Component

1. Double-clicktheJoinComponentintheFoundrypane. 2. NametheURISnapLogic/User/Exercise_3/JoinandclickOK. 3. SelecttheInputtabandclicktheAdd Inputbutton. 4. ClickNexttousetheViewCreationWizardtocopytheoutputview(Output1)fromyour FilterLeadsComponentfromExercise2.YoushouldhavethefieldsFirst,Last,Phone_w, Address,City,State,andZipdefined. 5. RepeatthesestepstocreateanInput2,copyingtheoutputviewfromyourCensusFeed Pipeline. Input2matchestheoutputviewoftheCensusReaderResource,Output001.Youshould havedefinedZip,LessThan10K,MoreThan10K,MoreThan25K,andMoreThan50Kfields. 6. ClickSave. 7. SelecttheOutputtabandcopyyourInput1inputviewandmanuallyaddtheMoreThan50Kfieldaftertheviewhasbeencopied. 8. SelectthePropertiestab. 9. ClickView/EdittosettheAliasessettings. 10. ClickAdd row.

- 27 -

Exercise 3: Data Services

11. Enterthefollowingvalues: l IntheFullyQualifiedFieldcolumn,selectInput1.Zip. l IntheOutputFieldcolumn,selectZip. 12. ClickClose. 13. ClickView/EdittosettheJoinExpressionssettings. 14. ClickAdd row. 15. Enterthefollowingvalues: l IntheJoinField1column,selectInput1.Zip. l IntheJoinField2column,selectInput2.Zip. 16. ClickClose. 17. ClickSave. 18. SelecttheGeneral Infotab.Fillintheauthoranddescriptionfieldsasdesiredand clickSave.Thesefieldsareoptional. TherearenoparametersdefinedforthisComponent,soyoucanleaveitunchanged. 19. ClickSave. TheJoinComponentisnowdefined.

Duplicating a Writer Component

TheProspectsComponentcreatedinyourExercise1folderisclosetowhatweneedfora Writerinthisexercise,soletsmakeacopyofthatversionoftheWriterandmodifyithere. 1. Double-clickontheProspectsComponentfromTutorialExercise1. 2. SelectSave AsfromtheSavemenubutton. 3. FromtheSaveResourceAs...dialog,selecttheURISnapLogic/User/Exercise_3/Prospects. 4. ClickOK. 5. SelecttheInputtab.BecausewecopiedthisComponent,weneedtoaddarowtoadd theMoreThan50Kfield.Thefieldtypeforthisfieldmustbenumber. DonotusetheDeletebuttonforeditingasthisdeletestheentireComponent.Ifyou wishtodeletearowfromtheoutputview,clickthe'''X'''buttontotheleftoftherow youwishtoremove. 6. SelecttheParameterstab.Again,sincewecopiedthisComponent,thevalueshere arealmostcorrectforourneeds.WejustwanttochangethedefaultvalueforOUTPUTFILEtobefile://tutorial/data/ex_3_prospects.csv. YourParameterstabshouldcontainthefollowinginformation.

- 28 -

SnapLogic Tutorials

Parameter Name DELIMITER OUTPUTFILE ,

Default Value file://tutorial/data/ex_3_prospects.csv

7. ClickSave. TheProspectsComponentisnowdefined.

Building a Data Service Pipeline

NextweneedtobuildaPipelinewiththetwoReaderComponentslinkedtotheJoinComponents,whichisinturnlinkedtotheWriterComponenttoperformajoin. 1. ClicktheNew Pipelinebutton. 2. IntheURIfield,enterSnapLogic/User/Exercise_3/EX3_Qual_CA_Prospects. 3. ClickOK. 4. DragtheLeadsComponentfromExercise1,theFilterLeadsComponentfromExercise 2,andtheJoinandProspectsComponentsandtheCensusFeedPipelinecreatedinthis exerciseontothePipelinecanvas.NametheComponentsasyoudropthemontothecanvastomatchtheirURInames. 5. ClickonthepurpleoutputviewofLeadsanddragtolinktoFilterLeads.TheFieldLinker dialogappearsandallfieldsshouldbeautomaticallylinked. 6. ClickOK. 7. LinkFilterLeadstoJoin. 8. SelecttheInput1inputview,thenclickOK.Againallfieldsshouldbeautomatically linkedintheFieldLinkerdialog. 9. LinkCensusFeedalsotoJoin. 10. SelecttheInput2inputviewandclickOK.AllfieldsshouldbelinkedintheFieldLinker dialog. 11. LinkJointoProspects. Autolinkingwasnotabletomatchallfields,somanuallymapanyremainingunlinked fieldsbyselectingtheappropriatefieldnamesfromthelistofvaluestotheleftofeach field. Forexample,fortheWork_Phonefield,manuallyselectPhone_wfromthelist. Whencompletethefieldsshouldbelinked. 12. FromthePipeline,selecttheParameterstab.

- 29 -

Exercise 3: Data Services

13. Edittheparameterdefinitionsasfollows. Pipeline Parameter LEADS CENSUS PROSPECTS Mapped to Leads.INPUTFILE CensusFeed.CENSUS Default Value file://tutorial/data/leads.csv file://tutorial/data/CAIncomeByZip.csv , , ,

Prospects.OUTPUTFILE file://tutorial/data/ex_3_prospects.csv

INPUT1_DELIM-Leads.DELIMITER ITER INPUT2_DELIM-CensusFeed.INPUT_ ITER DELIMITER OUTPUT_ DELIMITER Prospects.DELIMITER

14. NotethatwehavedistinguishedtheinputdelimitersasINPUT1_DELIMITER,INPUT2_ DELIMITER,andOUTPUT_DELIMITERtogiveustheoptionofdefiningthemuniquely shouldtheneedarise.Forthisexercise,allthedelimitersarecommas. 15. ClickSave. 16. SelecttheRuntab. 17. SelecttheRunsub-tab. ThePipelineParameterstableisnowpopulatedwiththeparameterstobeusedinthis Pipeline.Noticethatthedefaultvaluesappearinthetable.Ifnecessary,wecould changethevalueshere. 18. ClickRunbutton. ConfirmationthatthePipelineisstartingisprovidedonthestatuslineintheRuntime Informationsub-tab. VerifythatthePipelineperformedthistaskbyopeningtheoutputfileex_3_prospects.csv andverifyingthattheMoreThan50Kfieldhasbeenadded. WehaveaccomplishedthegoalofcreatingaDataServicePipelineandusingitaspartof anotherPipeline,whichtakesthedataintheoriginalleadsinputfile,refiningittoyieldjust theworkphonenumbersforalltheleadsandanewcolumnhasbeenpopulatedwithcensus dataforeachZipcode.(ThisnewcolumnshowsthepercentageofpeopleinaZipCodewho makemorethan$50,000ayear.)

- 30 -

SnapLogic Tutorials

- 31 -

5
Exercise 4: Data Sorting
Exercise4filtersandsortsjoineddatabasedonzipcodeswhere25%ormoreofhouseholds haveanincomegreaterthan$50,000.Theresultsarewrittentoafilenamedex_4_prospects.csv.

Reviewing the Filtered Qualifying California Prospects Pipeline

IntheLibrary,youwillseetheEx4_Filtered_Qual_CA_ProspectsPipeline.Letstakealookat theComponentsusedinthisPipeline,thenwellgocreateitourselves. Double-clicktheEx4_Filtered_Qual_CA_ProspectsPipelineintheLibrary.Thiswillopenthe PipelineinanewCanvastab.InadditiontoComponentsusedinpreviousexercises,youwill seetheFilterProspectsandSortProspectsComponents. Double-clickFilterProspectsandthenclick: l TheGeneral InfotabprovidesdescriptiveinformationforthisComponent.These fieldsareoptional. l ThePropertiestabiswhereyoudefinethefiltercriteriausedbythisComponent.In thisexample,wefilterforzipcodeswhereatleast25%ofhouseholdshaveincomes greaterthan$50,000.Todothis,wedefineafilterexpressiontocheckwhenthevalue oftheMoreThan50Kisgreaterthanorequalto0.25. l Intheseexercises,zipcodeisdefinedasastringdatatype. l TheInputtabdefinesthefieldsconsumedbythisComponent. l TheOutputtabspecifiesthefieldsoutputbythisComponent. l TheParameterstabshowsparametersusedbythisComponentaswellastheir optionaldefaultvalues. Double-clickSortProspects. l TheGeneral InfotabprovidesdescriptiveinformationforthisComponent.These fieldsareoptional. l ThePropertiestabiswhereyoudefinethesortingcriteriausedbythisComponent.In thisexample,wewillbesortingbasedonthefollowingfields:MoreThan50K(primary key),Zip_Code(secondarykey). l TheInputtabdefinesthefieldsconsumedbythisComponent. l TheOutputtabspecifiesthefieldsoutputbythisComponent.

- 32 -

SnapLogic Tutorials

l TheParameterstabshowsparametersusedbythisComponentaswellastheiroptional defaultvalues. ClickthePipeline Propertiesbutton. l TheGeneral InfotabprovidesdescriptiveinformationforthisPipeline.Thesefields areoptional. l TheInputtabdefinesthefieldsconsumedbythisPipeline.NoInputViewsareusedby thisPipeline. l TheOutputtabspecifiesthefieldsoutputbythisPipeline.NoOutputViewsareusedby thisPipeline. l TheParameterstabshowsparametersusedbythisComponentaswellastheir optionaldefaultvalues. TheCensus,Leads,FilterLeads,andJoinComponentsandusedinthisPipelineandtheirfield mappingsareidenticaltohowtheyaredefinedinExercise3.Thefollowinginformation focusesonwhathasbeenintroducedinthisexercise. ClickthelinkbetweentheJoinComponentandFilterProspectsComponent.Thisdisplaysthe followingfieldmappings:

NowclickthelinkbetweentheFilterProspectsComponentandSortProspectsComponent.You willseethefollowingfieldmappings:

Lastly,clickthelinkbetweentheSortProspectsComponentandProspectsComponent.This willshowthefollowingfieldmappings: RunningthisPipelineresultsinlistof112recordswrittentoafilenamedex_4_prospects.csv.TheserecordsexactlysatisfythecriteriathatthequalifyingleadsmustliveinCal-

- 33 -

Exercise 4: Data Sorting

iforniaandresideinzipcodeswithatleast25%ofhouseholdshavinganincomegreaterthan $50,000. NowthatyouvehadanopportunitytoreviewtheComponentsusedinthisexercise,lets createthisPipelinefromscratch.

Hands-on Exercise: Sorting Data

ThissectionexplainshowtobuildtheComponentandgeneratetheoutputcontainedinthe Tutorial/Exercise_4folders.Ifyouneedadditionalinformationatanytimeduringthis hands-onportionofthisexercise,feelfreetoopenandexamineeachoftheTutorialversions oftheseComponents.

Duplicating a Writer Component

1. OpentheEx3_Qual_QA_ProspectsPipeline. 2. Double-clicktheProspectsComponent. 3. Duplicate(SaveAs)theComponentasSnapLogic/User/Exercise_4/Prospects. 4. InthenewProspectsComponent,clicktheParameterstab. 5. ChangetheDefaultValuefortheOUTPUTFILEparametertoex_4_prospects.csv. 6. ClickSave. TheProspectsComponentisnowdefinedforExercise4.

Creating a Filter Component

1. IntheFoundry,typefilterintothesearchtextbox. 2. Double-clicktheFilterComponentTemplate. 3. SpecifytherelativeURIasSnapLogic/User/Exercise_4/FilterProspects. 4. ClickOK. 5. ClicktheInputtab. 6. ClickAdd InputtotriggertheAdd Input View Wizard. 7. ClickNext. 8. MakesurethattheCopy an existing viewradiobuttonisselected.ClickNext. 9. SelecttheOutputviewoftheJoinComponentcreatedinExercise3.ClickFinish. 10. ClickSave. 11. ClicktheOutputtab. 12. ClicktheAdd OutputbuttonandcopytheinputviewoftheFilterProspectsComponent. 13. ClickthePropertiestabandclicktheView/EditlinkforFieldFilters. 14. ClickAdd Row.

- 34 -

SnapLogic Tutorials

15. SpecifytheFieldFilterspropertiestobe: l Field:MoreThan50K l Value:0.25 l Comparison:>= l View:Output1 16. ClickClose. 17. ClickSave.

Creating a Sort Component

1. IntheFoundry,typesortintothesearchtextbox. 2. Double-clicktheSortComponentTemplateandcreateanewComponentwiththerelativeURItoSnapLogic/User/Exercise_4/SortProspects. 3. SelecttheInputtabandcopytheinputviewoftheProspectsComponent. 4. SelecttheOutputtabandcopytheinputviewoftheSortProspectsComponent. 5. ClickthePropertiestabandclicktheView/EditlinkforSortSpecifications. 6. ClickAdd Row. 7. SpecifytheSortspecspropertiestobe: l Sortfield:MoreThan50K l Sortorder:desc 8. ClickAdd RowandspecifytheSortspecpropertiestobe: l Sortfield:Zip_Cde l Sortorder:asc 9. ClickClose. 10. ClickSave.

Building the Pipeline

NextyoucannowlinktheseComponentstogether. 1. ClicktheNew Pipelinebutton. 2. SpecifytheURIasSnapLogic/User/Exercise_4/Ex4_Filtered_Qual_CA_Prospects. 3. ClickOK. 4. DragtherequiredComponentsontothePipelineCanvasandlinktheComponentsasfollows: l Census,connecttoJoin l Leads,connecttoFilterLeads

- 35 -

Exercise 4: Data Sorting

l FilterLeads,connecttoJoin l Join,connecttoFilterProspects l FilterProspects,connecttoSortProspects l SortProspects,connecttoProspects l Prospects

5. Definefieldmappingsasdescribedin"ReviewingtheFilteredQualifyingCaliforniaProspectsPipeline". 6. ClicktheParameterstabandmodifythePipelineParameterandDefaultfieldsfollows: Pipeline Parameter LEADS CENSUS PROSPECTS Mapped to Leads.INPUTFILE Census.INPUTFILE Prospects.OUTPUTFILE Default Value file://tutorial/data/leads.csv file://tutorial/data/CAIncomeByZip.csv file://tutorial/data/ex_4_prospects.csv , ,

INPUT1_DELIMITER Leads.DELIMITER INPUT2_DELIMITER Census.DELIMITER

OUTPUT_DELIMITER Prospects.DELIMITER , Wehavedistinguishedtheinputdelimitersas`INPUT1_DELIMITER`,`INPUT2_DELIMITER`,and`OUTPUT_DELIMITER`togiveustheoptionofdefiningthemuniquelyshould theneedarise.Forthisexercise,allthedelimitersarecommas. 7. ClickSave. 8. ClicktheRuntab. ThePipelineParameterstableispopulatedwiththeparameterstobeusedbythisPipeline.Noticethatdefaultvaluesappearinthetable.Ifnecessary,thesevaluescouldbe changedrightherebeforerunningthePipeline. 9. ClickRun. ThePipelinewillreadthedataspecifiedintheReaderCensusandLeadsComponents andwriteittothefilespecifiedintheProspectsComponent(ex_4_prospects.csv). Youcanverifytheresultsofthisexercisebyexaminingthat112recordscontainedin theex_4_prospects.csvfileallhaveanpercentagegreaterthan25%inthelastcolumn.

- 36 -

SnapLogic Tutorials

- 37 -

6
Exercise 5: Utilizing Pass-Through
Exercise5isbasedonExercise2andintroducesyoutotheconceptofpass-through.Componentsthatsupportpass-throughallowyoutomaponlythefieldsrequiredbytheComponent.Theremainingfields"pass-through"theComponentandareavailableforlinking downstream.SeethePass-throughsectionintheUserGuideformoreinformation. Note: TheTutorialPipelinesgeneratethesampleoutputfilesex_n_prospects.csv,(forexample,ex_1_prospects.csv,ex_2_prospects.csv).Whenyou runyourexercises,youwilloverwritethesefiles.Toavoidthis,youcanrename yourexerciseoutputfilestobeunique.Oryoucansimplyre-runthesamplepipelinestorecreatetheoutputfilesifyouneedtoexaminethem.

Reviewing the California Prospects Pipeline, Exercise 5

FromtheDataServericon(<my_machine_name>:8088),double-clicktheEx5_CA_Prospects folder.ThePipelineisidenticaltotheoneinExercise2withoneexception. DoubleclickFilterLeads.ThisComponentisalmostidenticaltotheoneusedinExercise2, exceptforthewaytheinputandoutputviewsareconfigured.Clickon: l TheInputtab.NoticethatonlytheStatefieldisdefined.SincethisfilterComponentis applyingapredicatetotheStatecolumn,Stateistheonlycolumnthatitreallyneedsas input. l TheOutputtab.AgainnoticethattheonlyfielddefinedistheStatefieldbutalsonotice thattheboxforviewInput1ischeckedinthePass-throughInputViewssection.This enablespassing-throughallthefieldsthataren'tlinkedtoafieldofInput1. BackontheEx5_CA_ProspectsPipeline,clickthelinkbetweentheLeadsResourcetotheFilterLeadsResourcetodisplaytheFieldLinkerpage. Inthisexercise,FilterLeadsonlyrequiresasinglefield,theState field. ClickanywhereonthelinkbetweentheFilterLeadsResourcetotheProspectsResourcetodisplaytheLinkmenuandselectField LinkingtodisplaytheFieldLinkerpage. Sincepass-throughhasbeenenabledinFilterLeads,notonlyistheStatefieldavailablebutall thefieldsthatwereavailabletotheinputviewofFilterLeads. WhenyouclicktheRun Pipelinebutton,thePipelineprocessestheReaderoutput,passesit throughtheFilterandintotheWriterinput,thencreatestheex_5_prospects.csvfile.You shouldseethattheoutputresultcontains675records. NowletscreatetheResourcesinExercise5.

- 38 -

SnapLogic Tutorials

Hands-on Exercise: Implementing Pass-Through

Creating a Filter Component with Pass-Through
1. Double-clicktheFilterComponentintheComponentLibrarypane. 2. SpecifytheURIasSnapLogic/User/Exercise_5/FilterLeads. 3. ClickOK. 4. SelecttheInputtab,clicktheAdd View button,clickNextwheretheinputcanbecopiedfromanexistingvieworcanbecreatedasanewview.SinceourComponentonly needsonefield,wewillselectthenewblankviewoptionandclickFinish. 5. ClickAdd Rowtoaddasinglerow. 6. NamethefieldState.Thedatatypeshouldbestringandaddafielddescription. 7. ClickSave. 8. ClicktheOutputtab. 9. RepeatthestepsyouperformedtodefinetheinputviewInput1 tocreatetheoutput viewOutput1,orusethecopyviewwizardtocopythedefinitionofInput1. 10. ChecktheboxnexttoInput1 toenablepass-throughonthisoutputview. 11. ClickSave. 12. ClickthePropertiestabfortheFilterLeadsComponent. 13. Specifythepropertiesexactlyasyoudidinexercise2. 14. ClickonView/EditlinkofFieldFilters,thenaddtherow: l Field:NameoftheCSVfilecolumnisState l Operator:== l Value:ThevaluetofilteronisCA l View:Output1 15. ClickClose. 16. ClickSave. TheFilterisnowdefined.

Duplicating the Reader and Writer Resources

DuplicatetheLeadsandProspectsResourcescreatedinyourExercise2folder. 1. Double-clicktheLeadsResourceiconfromTutorialExercise2toopenit. 2. SelectSave AsfromtheSavemenubuttondrop-downlist. 3. FromtheSave Resource Asdialog,specifytheURIasSnapLogic/User/Exercise_ 5/Leads.

- 39 -

Exercise 5: Utilizing Pass-Through

4. ClickOK. TheduplicatedLeadsResourceappearsinSnapLogic > User > Exercise_5. SincetheLeadsReaderwillfunctionexactlyasitdidinExercise2(thatis,readingin thefieldsfromtheleads.csvfile)wecanleaveitsProperties,Outputview,andParametersunchanged. 5. PerformthesamestepsasaboveforduplicatingtheProspectsresourcefromExercise 2.SpecifytheURIintheSaveAsdialogtobeSnapLogic/User/Exercise_ 5/Prospects.

Building the Pipeline

NowwecanlinktheLeads,Prospects,andFilterLeadsResourcestogetherinaPipelinejustas inExercise2toprocessthedata. 1. ClicktheNew Pipelinebutton. 2. SpecifytheURIasSnapLogic/User/Exercise_5/CA_Prospects. 3. ClickOK. 4. DragtheFilterLeadsComponentontothePipelinecanvas.Notethat_1isappendedto theComponentname. 5. RenametheComponentFilterLeads. 6. DragyourLeadsandProspectsResourcesontothePipelinecanvas.NamethemLeads andProspectsrespectivelyasyoudropthemontothecanvas. 7. LinktheLeadsReadertotheFilterLeadsResource. 8. Bydefault,theFieldLinkerdialogappears,andtheStatefieldswillbemappedautomaticallybecauseamatchingfieldnamewasfound. 9. ClickApplyandSavewhenfinished. 10. LinktheFilterLeadsResourcetotheProspectsWriter. 11. ClickApplyandSavewhenfinished. 12. FromthePipeline Properties,selectthePipeline Parameterstab. 13. Edittheparameterstochangethedefaultnames. 14. ClickSave. 15. ClicktheRun PipelinefromthePipelinemenu. ThePipelinewillexecuteexactlyasitdidinExercise2.Theonlydifferenceisthatthefields unneededbytheFilterresourcearepassedthroughtothe"downstream"resource. Openex_5_prospects.csvandverifythefilteredresults.Youshouldseeonlythe675leads thathaveCaliforniaaddresses,similartotheoutputfromExercise2.

- 40 -

SnapLogic Tutorials

- 41 -

Index
C
Components creating 8 data service 25 filter 17, 34 filter with pass-through 39 join 27 sort 35 writer 11 duplicating 19, 28, 34, 39

E
exercises overview 6, 16, 22, 32, 38

P
Pipelines building 12, 20, 29, 35, 40

T
tutorials environement 4 overview 5

- 42 -

SnapLogic Tutorials

- 43 -