Are you looking to join a fast-paced, highly-inventive Data Engineering team helping to build earth's largest transportation network?
As a Data Engineer on the Surface Transportation team, you will be working in a large, complex, and dynamic data environment. We are looking for a highly skilled and motivated Data Engineer with an uncanny ability to integrate multiple heterogeneous data sources to build efficient, highly available, secure, and scalable data warehouse and reporting solutions. The ideal candidate has deployed analytics platforms to 1000+ users, optimizing the architecture for performance and positive end-user experience. You can effectively evaluate the pros and cons of different design decisions, and make appropriate architectural trade-offs. You are enthusiastic about learning new technologies and implementing solutions using these technologies to empower internal customers and scale the platform. You demonstrate solid communication skills and the ability to partner with both technical and non-technical teams to build data solutions that enable the business to gain insights and make decisions.
Key job responsibilities
- Design, build, and maintain batch and streaming data pipelines using modern Big Data technologies such as AWS Redshift, S3, Glue, Lake Formation, Athena, Kinesis, Lambda, etc.
- Manage data infrastructure as code using AWS CloudFormation and the AWS Cloud Development Kit (CDK)
- Automate processes using scripting languages such as Python, Scala, etc.
- Optimize, tune, and govern data warehouses and data lake environments to ensure performance for large user bases
- Work collaboratively with software engineers, data scientists, business analysts, and other internal partners to identify opportunities and build scalable solutions
- Possess strong verbal and written communication skills, be self-driven, and deliver high quality results in a fast-paced environment
