Job Description
About the Company: Leading technology firm
Experience:5 years
Location :Hyderabad
Job Title: Data Engineer
Roles and Responsibilities:
Data Engineer builds and maintains systems that collect, store, and process large volumes of dataDesigns medallions data architectures like Lakehouse with Bronze, Silver, Gold layers for scalable data processing.Uses Apache Spark to process large datasets efficiently Experience with Azure/AWS/GCP data platformsOptimizes data pipelines for speed, performance, and costCombines data from multiple sources into clean, consistent formats.Develops batch and real-time (streaming) data pipelinesImplements incremental data processing and data versioningAutomates workflows using CI/CD tools and DevOps practicesMonitors pipelines with logging, alerts, and error handlingWorks with cloud platforms and orchestra...