Engineer II, Big Data

Pilot Flying J is the 10th largest privately held company in North America with more than 28,000 team members. As the industry-leading network of travel centers, we have more than 950 retail and fueling locations in 44 states and six Canadian provinces. Our energy and logistics division is a top supplier of fuel, employing one of the largest tanker fleets and providing critical services to oil operations in our nation's busiest basins. Pilot Company supports a growing portfolio of brands with expertise in supply chain and retail operations, logistics and transportation, technology and digital innovation, construction, maintenance, human resources, finance, sales and marketing.

Founded in 1958, we are proud to be family owned and consider our team members to be part of the family. Our founding values, people-first culture and commitment to giving back remains true to us today. Whether we are serving guests, a fellow team member, or a trucking company, we are dedicated fueling people and keeping North America moving.

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or protected veteran status.

Pilot Flying J is part of the Pilot Company family of brands that keeps North America's drivers moving, including E-Z Trip, Mr. Fuel, One9 Fuel Stop, Pride, StaMart and Xpress Fuel.

Every single day we are proud to make life better for our team members and our guests. We are fueling life's journeys and a career with Pilot is an opportunity to work hard, have fun and live up to your full potential. Do you want to become part of the Pilot family?

As a Data Engineer, you will work closely with a cross-functional Agile team to build high quality data pipelines driving analytic solutions. These solutions will generate insights from our connected data, enabling Pilot to advance our data-driven decision-making capabilities.

Furthermore, you will:

1. Research, develop, document, and modify Big Data Lake processes and jobs according to data architecture and modeling requirements, and processes set forth by the senior Business Intelligence (BI) data team members
2. Maintain accountability for completing the key EDW/Big Data Lake project activities; communicate project status and risks to project leader
3. Troubleshoot issues including connection, failed jobs, application errors, server alerts, and space thresholds within predefined service level agreements (SLAs)
4. Proactively maintain and tune SQL code according to EDW and Big Data Lake best practices
5. Review and ensure appropriate documentation for all new development and modifications of the big data processes and jobs
6. Perform unit testing for solutions developed, and ensure integrity and security of institutional data
7. Take technical specifications and complete development of data solutions for analytics
8. Take business specifications for small to medium projects/data requests and create technical specifications, and data models
9. Perform code reviews of work from other team members
10. Mentor junior team members in best practices
11. Model behaviors that support the companys common purpose; ensure guests and team members are supported at the highest level
12. Ensure all activities are in compliance with rules, regulations, policies, and procedures of the organization
13. Complete other duties as assigned

• Bachelors degree in computer science, business administration, information technology, or related field required
• Minimum three years experience in technology operations required; five years experience preferred
• Strong SQL knowledge and skills required
• Familiarity with enterprise AWS cloud environment or similar data platform with an emphasis in EMR and Redshift required
• Experience with Spark/Spark-streaming, Python required
• Knowledge of Postgres and SQLServer preferred
• Knowledge of ETL tools such as Informatica or Datastage preferred
• PFJ institutional knowledge preferred
• Excellent analytical skills and the ability to identify solutions to complex data problems
• Ability to communicate and interact effectively with team leadership and different levels of the organization
• Strong customer service skills
• Attention to detail and accuracy
• Strong written and verbal communication skills
• Willingness to learn and embrace new technologies
• Ability to prioritize and execute tasks in a fast-paced and dynamic environment
• Ability to take initiative, work independently and with a team
• Travel required less than 10%
• General office work requiring sitting or standing for long periods of time
• Participate in a 24/7 on-call rotation for after-hours and weekend production support

• Nationwide Medical Plan/Dental/Vision
• 401(k) and Flexible Spending Accounts
• Adoption Assistance
• Tuition Reimbursement
• Weekly Pay
• All your information will be kept confidential according to EEO guideline