01/2021–present SeniorLead DataEngineer
InvolvedinBusinessrequirementdocumentwalkthroughandclientcallstounderstandfunctionality
DesignedastablearchitectureforProject
Involved in planning and development of ETL pipelines using ADFUnblockingtheteammembersinrequirementsandtechnicalissuesDesignedAzureDataFactoryPipelinestomovedatatoAzureDataLakeImplementedADFPipelinestopulldatafromdifferentsources
Accessed Azure Data Lake files data in Azure Data bricks using mount pointsImplementedBusinesslogicsinSparkSQL/PysparktopopulaterequiredtablesHandledcleansing,aggregationsandtransformationsinSpark
ImplementedAzureFunctionswithPythontopulldatafromAPIsandscheduledtheseAzureFunctionsinAzureDataFactory
ImplementedendtoendloggingframeworkforAzureDatafactorypipelineDesignedofDatawarehouseforstableimplementation.
IntegratedAzureDataFactoryforschedulingJobsandTriggeringotherrelatedJob.ExtensivelyusedAzureKeyVaultstoconfiguretheconnectionsinLinkedservicesCreatedLogicAppstotriggerthemailnotificationfromADFPipelines.
ImplementedStoredProceduresbusinesslogicinAzureSQLDatawarehouse.
CreatedCI/CDpipelinesforADF,AzureSQLDatawarehouseandAzureSQLDatabasedeploymentinADO
DeployedPipelineandcodetomultipleenvironmentsusingCI/CDpipelineValidatedthedataatoverallandeverydimensionlevel.
WrittentestcasestovalidatedatainADO
09/2018–12/2020 SeniorSoftwareEngineer
DesignedofDatawarehouseforstableimplementation.
Designed anddevelopedAzureDatafactorypipelines forDatamigration.IntegratedAzureDataFactoryforschedulingJobsandTriggeringotherrelatedJob.
Created LogicApps topull thedata fromSharePoint excelsto ADLS.ImplementedAzureDatabricksnotebooksbusinesslogicatDatawarehouselevel.CreatedamountpointinPythontoaccessthefilesresideinADLS.
CreateduserdefinedfunctionsusingPython.
Createddataframes,viewsandtablesusingPythoninAzureDatabricks.
Used Python loops and aggregation techniques to process data in Azure Databricks.PublishedtheFactandDimensiontablesfromADBtoAzureSQLDatawarehouseandADLSusingPython.
CreatedvalidationframeworkusingPythoninAzureDatabricks.
CreatedTabularmodelonFactandDimensionviewswhichareinAzureSQLDatawarehouse.
UpdatedTabularlevelmeasuresanddimensions,wroteDAXqueriesforcalculatedmeasures.
ValidatedthedataatoverallandeverydimensionlevelinTabular,checkedondataisslicingateverydimensionlevel
WrittentestcasestovalidatedatainatTabularinADODevelopedreportsusingPowerBI.
07/2017–08/2018 SoftwareEngineer2
Designedanddevelopedrelationaldatabases.Guidingteammembers.
DevelopedSSISpackagestoloadthedatafromdifferentsourcestoSQLandcreateaDatawarehouse.
ImplementedStoredProcedurestoprocessbusinesslogicatDatawarehouselevel.CreatedTabularmodelontopofDatawarehouse
ImplementedmainbusinesslogicforfinancialmetricsinTabularusingDAXqueriesValidatedthedataatoverallandeverydimensionlevelinTabular,checkedondataisslicingateverydimensionlevel
DevelopedreportsusingPowerBI.
WroteautomatedBVTSPstocheckvaluesateachlevelonrefreshofdatadevelopedscriptfilesonbigDataplatformtomaketheheavydatatobeprocessed
Copyright© Cosette Network Private Limited All Rights Reserved