DATA ENGINEER I or II
The Enterprise Decision Support and Data Governance (EDGE) Innovation team is seeking a highly skilled Data Engineer who will also serve as a Data Subject Matter Expert (SME). In this role, you will play a crucial part in developing and maintaining the enterprise curation layer within the EDGE. You will leverage your extensive experience in data engineering to ensure the data within EDGE is accessible, reliable, and high performing for our utilization by our Innovation, data & analytics teams throughout the enterprise.
This position follows a hybrid work schedule. Employees are required to be in the office four (4) days per week.
This position may be filled as an Engineer I (GSE05) or Engineer II (GSE06). Individuals who demonstrate advanced skills in terms of work experience, education, and certifications may be eligible for the more advanced position.
SUMMARY
The Data Engineer will collaborate with business and technology stakeholders to understand requirements, define data models, and implement Gas data assets. They will manage data governance and maintain a data catalog. The role includes supporting machine learning model creation, training, operation, and automation. Additionally, they will enforce data standards and collaborate with various teams to deploy optimized data solutions, plan capacity, and ensure secure data storage and usage.
MAJOR JOB RESPONSIBILITIES:
Functional Expertise:
- Provide key knowledge, insight, and informational guidance related to the integration of data and design of the holistic data landscape in support of the EDGE organization’s objectives
- Translate Decision Support Requirements related to KPI’s, Indices, and other analytical insights into Executable strategies, approaches and platform designs or other means of information delivery
- Produce & Manage all Decision Support Solution Design Frameworks and their supporting Documentation
- Identify and propose new capabilities and improvements to Decision Support capabilities
- Guide, Advise and Support other teams on implementation issues related to application of data related capabilities and approaches
- Perform expert oversight and guidance for delivery of assigned Decision Support Solutions
- Support Business Information Architects in the communication and interpretation of information and analysis to business leadership
Responsibilities
- Support EDGE Project related assignments as required
- Coordinates with the TO to identify approaches and solutions to data implementation problems related to the supporting EDGE solutions platforms
- Develop and enhance the data management processes within the curation layer of the EDGE platform, ensuring scalable and accurate data handling and retrieval.
- Create data models and construct efficient data pipelines, ETL processes, and workflows using Databricks, optimizing data ingestion transformation, and storage to support analytical needs.
- Collaborate with data scientist to refine data collections methods while building and tuning Databricks jobs and Apache Spark application for improved performance and scalability.
- Implement best practices for data management practices including partitioning, indexing, and compression to increase query efficiency.
Business Acumen:
- Evaluates and recommends approaches and strategies on deployment and application of the tools and technologies adopted by Southern Company with the TO in the development and delivery of solution strategy, platforms, and applications
- Develop and strengthen data standards and policies to maintain data integrity and consistency across the EDGE.
- Adhere to and implement data governance protocols to uphold high-security standards and clear data lineage.
- Proactively monitor data systems to identify and resolve issues that could impact data accuracy.
Engagement:
- Collaborates with other team members, TO and business managers to resolve complex business issues and problems
- Develops and maintains productive working relationships with key EDGE and TO managers and staff
- Engage with data scientists, analysts, and business stakeholders, to precisely understand their data needs and translate these into Databricks-based technical solutions.
- Offer technical guidance and support, enabling the seamless introduction and adoption of advanced data solutions.
- Remain abreast of the latest data engineering trends, particularly in Databricks and other cloud-based technologies.
DATA ENGINEER I
Education, Certifications/Licenses:
Required:
- Bachelor's degree in related discipline (e.g., Computer Science, Information Management or equivalent) or equivalent related work experience
Preferred:
- Master's degree in related discipline (e.g., Computer Science, Information Management or equivalent) or equivalent related work experience
Related Work Experience:
Required:
- 1 - 3 years data-oriented solution architecture development experience utilizing Both On-Premises and Cloud (Azure) modern data architecture platforms and patterns.
- 3 years of experience developing context diagrams as well as conceptual, logical, physical architecture, data model and ER diagrams as well as data orchestration diagrams.
- 2 years hands-on experience designing and implementing data models for structured and unstructured datasets utilizing tools such as ERWIN, SQL Developer, Python, SSMS, Visual Studio and Visio.
- Ability to deliver results in a cross functional team environment
Preferred:
- Expert knowledge and hands-on experience working with SQL.
- Hands on experience working with relational databases such as Oracle, SQL Server as well as Azure Data Lake, Synapse Data Warehouse.
- Experience in Data Governance processes and tools such as Informatica Data Catalog
Specific Skills & Knowledge:
Required:
- Advanced proficiency in MS Office Tools
- Experience in Data Quality processes and tools such as DQS
- Hands-on experience developing big data technologies such as Distributed file Systems, HIVE and Spark
- Ability to provide guidance on app/data performance, high availability and disaster recovery
- Experience with designing re-usable solution components and perform data model integrity checks
- Experience working in a medium to large enterprise setting and collaborating multiple cross-functional teams
- Experience working on concurrent engineer assignments on multiple projects.
- Significant knowledge of business requirements and solutions design methodologies, tools, and approaches
- Significant knowledge of graphical modeling approaches, tools, and repositories
- Significant knowledge of Logical and Physical Data Modelling approaches, tools, and methodologies, and supporting software platforms
DATA ENGINEER II
Education, Certifications/Licenses:
Required:
- Bachelor's degree in related discipline (e.g., Computer Science, Information Management or equivalent) or equivalent related work experience
Preferred:
- Master's degree in related discipline (e.g., Computer Science, Information Management or equivalent) or equivalent related work experience
Related Work Experience:
Required:
- 3 - 5 years data-oriented solution architecture development experience utilizing Both On-Premises and Cloud (Azure) modern data architecture platforms and patterns.
- 3- 5 years of experience developing context diagrams as well as conceptual, logical, physical architecture, data model and ER diagrams as well as data orchestration diagrams.
- 2- 4 years hands-on experience designing and implementing data models for structured and unstructured datasets utilizing tools such as ERWIN, SQL Developer, Python, SSMS, Visual Studio and Visio.
- Ability to deliver results in a cross functional team environment
Preferred:
- Expert knowledge and hands-on experience working with SQL.
- Hands on experience working with relational databases such as Oracle, SQL Server as well as Azure Data Lake, Synapse Data Warehouse.
- Experience in Data Governance processes and tools such as Informatica Data Catalog
Specific Skills & Knowledge:
Required:
- Advanced proficiency in MS Office Tools
- Experience in Data Quality processes and tools such as DQS
- Hands-on experience developing big data technologies such as Distributed file Systems, HIVE and Spark
- Ability to provide guidance on app/data performance, high availability and disaster recovery
- Experience with designing re-usable solution components and perform data model integrity checks
- Experience working in a medium to large enterprise setting and collaborating multiple cross-functional teams
- Experience working on concurrent engineer assignments on multiple projects.
- Significant knowledge of business requirements and solutions design methodologies, tools, and approaches
- Significant knowledge of graphical modeling approaches, tools, and repositories
- Significant knowledge of Logical and Physical Data Modelling approaches, tools, and methodologies, and supporting software platforms
About Southern Company Gas
Southern Company Gas is a wholly owned subsidiary of Atlanta-based Southern Company (NYSE:SO), America’s premier energy company. Southern Company Gas serves approximately 4.2 million natural gas utility customers through its regulated distribution companies in four states and approximately 600,000 retail customers through its companies that market natural gas. Other nonutility businesses include investments in interstate pipelines and ownership and operation of natural gas storage facilities. For more information, visit southerncompanygas.com .
Southern Company (NYSE: SO ) is a leading energy provider serving 9 million customers across the Southeast and beyond through its family of companies. Providing clean, safe, reliable and affordable energy with excellent service is our mission. The company has electric operating companies in three states, natural gas distribution companies in four states, a competitive generation company, a leading distributed energy solutions provider with national capabilities, a fiber optics network and telecommunications services. Through an industry-leading commitment to innovation, resilience and sustainability, we are taking action to meet customers' and communities' needs while advancing our goal of net-zero greenhouse gas emissions by 2050. Our uncompromising values ensure we put the needs of those we serve at the center of everything we do and are the key to our sustained success. We are transforming energy into economic, environmental and social progress for tomorrow. Our corporate culture has been recognized by a variety of organizations, earning the company awards and recognitions that reflect Our Values and dedication to service. To learn more, visit www.southerncompany.com .
Southern Company invests in the well-being of its employees and their families through a comprehensive total rewards strategy that includes competitive base salary, annual incentive awards for eligible employees and health, welfare and retirement benefits designed to support physical, financial, and emotional/social well-being. This position may also be eligible for additional compensation, such as an incentive program, with the amount of any bonus/awards subject to the terms and conditions of the applicable incentive plan(s). A summary of the benefits offered for this position can be found here https://seo.nlx.org/southernco/pdf/SOCO-Benefits.pdf . Additional and specific details about total compensation and benefits will also be provided during the hiring process.
Southern Company is an equal opportunity employer where an applicant's qualifications are considered without regard to race, color, religion, sex, national origin, age, disability, veteran status, genetic information, sexual orientation, gender identity or expression, or any other basis prohibited by law.
Job Identification: 13192
Job Category: Data Analytics
Job Schedule: Full time
Company: AGL Services Company
