Here are some sample job advertisements for different types of Data Engineer roles:
- Design, construct, install, test and maintain highly scalable data pipelines with state-of-the-art monitoring and logging practices.
- Bring together large, complex and sparse data sets to meet functional and non-functional business requirements.
- Design and implement data tools for analytics and data scientist team members to help them in building, optimizing and tuning our product.
- Integrate new data management technologies and software engineering tools into existing structures.
- Help in building high-performance algorithms, prototypes, predictive models and proof of concepts.
- Use a variety of languages, tools and frameworks to marry data and systems together.
- Recommend ways to improve data reliability, efficiency and quality.
- Collaborate with Data Scientists, DevOps and Project Managers on meeting project goals.
- Tackle challenges and solve complex problems on a daily basis.
- Cross-channel customer engagement strategy, design and development
- (web, mobile, social, physical)
- eCommerce strategy, implementation and operations
- Marketing Content and digital asset management solutions
- Marketing Technology and Advertising Technology solutions
- Marketing analytics implementation and operations
- Advertising campaign ideation, development and execution
- Acquisition and engagement campaign ideation, development and execution
- Agile based, design-thinking, user-centric, empirical projects that accelerate results
8+ years of experience in software development, a substantial part of which was gained in a high-throughput, decision-automation related environment.
4+ years of experience in working with big data using technologies like Spark, Kafka, Flink, Hadoop, and NoSQL datastores.
3+ years of experience on distributed, high-throughput and low-latency architecture.
1+ years of experience deploying or managing data pipelines for supporting data-science-driven decisioning at scale.
A successful track-record of manipulating, processing and extracting value from large disconnected datasets.
Producing high-quality code in Python.
Passionate about testing, and with extensive experience in Agile teams using SCRUM; you consider automated build-and-test to be the norm.
Proven ability to communicate in both verbal and writing in a high performance, collaborative environment.
Follows data development best practices, and enjoy helping others learn to do the same.
An independent thinker who considers the operating context of what he/she is developing.
Believes that the best data pipelines run unattended for weeks and months on end.
Familiar with version control, you believe that code reviews help to catch bugs, improves code base and spread knowledge.
Ability to travel 5-10% of the time
Experience with large consumer data sets used in performance marketing is a major advantage.
Familiarity with machine learning libraries is a plus.
Well-versed in (or contributes to) data-centric open source projects.
Reads Hacker News, blogs, or stays on top of emerging tools in some other way
Industry-specific marketing data
Technologies of Interest:
Languages/Libraries – Python, Java, Scala, Spark, Kafka, Hadoop, HDFS, Parquet.
Cloud – AWS, Azure, Google
At Disney Streaming Services, data is central to measuring all aspects of the business, and critical to its operations and growth. The data engineering team is responsible for collecting, analyzing and distributing data using public cloud and open source technologies and offers transparency into customer behavior and business performance.
If you are interested in joining Disney Streaming Services in the pursuit of not only crafting new media products but enjoying the products you build, we are interested in hearing from you.
- Collaborate with product teams, data analysts and data scientists to design and build data-forward solutions
- Design and build and deploy streaming and batch data pipelines capable of processing and storing petabytes of data quickly and reliably
- Integrate with a variety of data metric providers ranging from advertising, web analytics, and consumer devices
- Build and maintain dimensional data warehouses in support of business intelligence tools
- Develop data catalogs and data validations to ensure clarity and correctness of key business metrics
- Drive and maintain a culture of quality, innovation and experimentation
- Coach data engineers best practices and technical concepts of building large scale data platforms
Basic Qualifications :
- 3-5 years of experience developing in object oriented Python
- Experience deploying and running AWS-based data solutions and familiar with tools such as Cloud Formation, IAM, Athena, and Kinesis
- Experience engineering big-data solutions using technologies like EMR, S3, Spark and an in-depth understanding of data partitioning and sharding techniques
- Familiar with metadata management, data lineage, and principles of data governance
- Experience loading and querying cloud-hosted databases such as Redshift and Snowflake
- Building streaming data pipelines using Kafka, Spark, or Flink
- Familiarity with binary data serialization formats such as Parquet, Avro, and Thrift
- Experience deploying data notebook and analytic environments such as Jupyter and Databricks
- Knowledge of the Python data ecosystem using pandas and numpy
- Experience building and deploying ML pipelines: training models, feature development, regression testing
- Experience with graph-based data workflows using Apache Airflow
Required Education :
Bachelor’s degree in Computer Science or related field or equivalent work experience
We are looking for someone who is passionate about data and who enjoys solving challenging problems by coming to understand and utilize complex datasets. You will be working closely with the data strategy team on high-impact revenue-generating and cost-saving projects for the business, specifically ensuring stakeholders have the data they need for analysis and can trust the integrity of the data. You will be responsible for setting up and maintaining critical data pipelines into our big data environment along with marrying various complex datasets to support analyses. You will also be a key contributor in moving our on-premise big data environment to the cloud.
- Build critical data ingestion pipelines for new datasets required to support organization-wide analytics’ needs, while ensuring data integrity is well-maintained
- Support day-to-day activities of the data strategy team responsible for key revenue-driving or cost-saving analyses, specifically mining for critical datasets and/or taking the lead on complex analyses
- Help drive self-service analytics throughout the organization by capturing and organizing important data and/or domain knowledge in a centralized location
- Support our cloud-migration effort, moving our on-premise big data environment to the cloud
- You have a bachelor’s degree in computer science or equivalent experience
- 1-2 years of hands-on experience using SQL
- You are by nature a curious individual and a lifelong learner
- You are meticulous and cautious in how you approach a challenge, ensuring that you check and document your work along the way
- You find it fun thinking critically and creatively with others
- You have a passion for data and find yourself using it to defend your position or to support a new decision
- You love music and enjoy the idea of supporting music creators
- Strong SQL knowledge
- Experience in Python and/or a similar coding language
- Excellent verbal and written communication skills
- Exposure to Tableau or similar reporting/visualization tools
What We Love About You:
- Curious: You are a lifelong learner and are driven to answer unanswered questions
- Hands-On: You are willing to get your hands dirty in order to accomplish the task at hand. You have a personal mantra of “there’s always a way…”
- Data-Driven: You leverage data to support your opinions and find that more detail is always preferable.
- Honest: You’re willing to admit you don’t know the answer or that you have made a mistake. You consider analysis a moral endeavor and strive to prepare truthful output independent of any potential consequence.
- Audible: You speak up when you disagree or don’t understand something. There are no rockstars. We want to hear what you think.
- Master of Your Craft: You take pride in your work and strive to learn more to hone your skills.