Big Data Developer

Role: Big Data EngineerLocation: RemoteDuration: Long Term

Responsibilities:• Streamline the intake of the raw data into our Azure Data lake.• Perform production support and deployment activities.• Proactively drive the execution of our core data engineering, business intelligence, and data warehouse frameworks• Build data pipelines from systems such as CRM, Ecommerce etc. with the emphasis on scalability and reliability• Leverage central data warehouse along with other data sources to create enriched customer information into CRM system• Understand and translate business needs into data models to support long-term, scalable, and reliable solutions• Create logical and physical data models using best practices to ensure high data quality and reduced redundancy• Drive data quality across the organization; develop best practices for standard naming conventions and coding practices to ensure consistency of data models and tracking• Define and manage SLA’s for data sets and processes running in production• Continuously improve our data infrastructure and stay ahead of technology• Design a system for data backup in case of system failure• Build strong cross-functional partnerships with Data Scientists, Analysts, Product Managers and Software Engineers to understand data needs and deliver on those needs

We are excited about you if you have:• 10+ years of experience with large scale data warehouse projects .• Advanced Proficiency in Scala, Spark and SQL.• Excellent knowledge in data structures and design patterns• Preferred experience with data integration tools• Experience working with Retail/CRM/Finance datasets preferred• Knowledge in designing facts, dimensions, snapshots, SCDs, etc• Write SQL for processing raw data, data validation and QA• Knowledge working with APIs to collect or ingest data• Strong Database knowledge, SQL & No-SQL preferred• Communication Skills Data Engineers are part of a team, working with database administrators, data analysts and management and need to be effective communicators.• Attention to Detail Databases are complex, and a minute error can cause huge problems.• Problem-Solving Skills Data Engineers look at an issue that needs to be solved and come up with solutions quickly.• Experience designing and implementing cloud based and SaaS data warehouse (e.g. WS, Hadoop, NoSQL) and developing ETL/ELT pipelines• Experience integrating and building data platform in support of BI, Analytics, Data Science, and real-time applications• Strong communication skills, with the ability to initiate and drive projects proactively and accurately

RaghavaEmail-id: raghava@cygnuspro.com