Data Architect / engineer - Hands-on - REMOTE - MA

• Proficiency in SQL and  Python 
• Must have experience with Airflow/Composer.
• Experience with tools like DBT and Airbyte/Fivetran
• Must have experience with cloud data warehouse solutions like BigQuery/Snowflake/RedShift/Databricks

The Tech lead /  Data Architect will play a key role in in the Design and Development of Data models, Data pipelines, datasets, and ETL/ELT pipelines from a variety of data sources to the Data Warehouse / Data Lake on Google Cloud Platform (GCP) for our flagship product. It relies heavily on this foundational data layer.

The Data Architect will help provide technical leadership and strategic direction for Priority Engine’s data platform. The position will be responsible for the delivery and technical quality of our programs, taking on complex projects. It will also partner with cross-functional stakeholders to drive initiatives forward, and mentor team members in technical design, project leadership, and running team processes. The position will be responsible to build and maintain scalable data infrastructure

The candidate must have a keen sense for the business drivers, have a vision for the data strategy and help execute it. The candidate must be familiar with industry trends and best practices around data engineering on the cloud and apply them appropriately. The candidate must be a hand-on professional and able to explain work experiences with these technologies ( knowledge or certificates without actual experience is not acceptable.)

Responsibilities:

• Cloud Data Strategy: Develop and implement a comprehensive cloud data strategy, aligning it with the organization's business objectives, ensuring scalability, reliability, and security
• Data Architecture Design: Design and implement the cloud-based data architecture, considering factors such as data modeling, data integration, data storage, and data governance. Evaluate and select appropriate cloud technologies and services to support the architecture, such as data lakes, data warehouses, and data pipelines
• Data Modeling and Design: Define data models, schemas, and structures to effectively capture and represent data requirements. Ensure data models are optimized for performance, scalability, and ease of use.
• Data Integration: Develop strategies and implement solutions for seamless data integration from various sources into the cloud-based data architecture. Collaborate with cross-functional teams, including data engineers and data analysts, to design and implement data pipelines and ETL processes.
• Cloud Infrastructure Optimization: Collaborate with infrastructure teams to optimize the cloud environment for data storage, processing, and analytics. Identify opportunities to leverage cloud-native services and technologies to enhance performance and cost efficiency
• Collaboration and Stakeholder Management: Work closely with business stakeholders, data analysts, and other teams to understand data requirements and translate them into actionable solutions. Provide guidance and technical leadership to cross-functional teams to ensure alignment with the cloud data architecture
• Design, develop, and implement end-to-end data solutions (storage, integration, processing, access) in Google Cloud Platform (GCP);- Any other cloud technology is fine - AWS or Azure
• Create ETL/ELT pipelines that transform and process terabytes of structured and unstructured data in real-time;
• Deeply immerse in our Data Pipelines, to acquire full understanding and ability to converse with business representatives and individual contributors
• Some upcoming technical challenges include scaling our data ingestion pipelines across a growing number of GCP, AWS, and data center-based data sources, reducing the latency of our product data ingestion pipelines through moving batch jobs into a streaming architecture, and extending our data lake architecture for the growing ecosystem of data ingestion and creation tools.
• Be a hands-on contributor on the team;
• Collaborate with product/application architects to develop holistic solutions;
• Implement operational procedures (logging, monitoring, alerting, etc.) for dependable running of pipelines/jobs; and
• Be autonomous. You own what you work on. You move fast, take ownership, and get things done. The Data lake/warehouse is a central and essential service for multiple initiatives, and you need to be able to multitask and juggle priorities on a regular basis with a deep understanding of business impacts of prioritization decisions.

Requirements:

• BS Degree in Computer Science or a related field
• Proven experience as a Cloud Data Architect or in a similar role, with a focus on designing and implementing cloud-based data architectures.
• 10+ years industry experience, 4+ years in a similar role
• Strong experience of cloud platforms (e.g., AWS, Azure, Google Cloud) and their data services.
• Expertise in data modeling, database design, and data integration techniques.
• Proficiency in SQL and Python and scripting languages for data manipulation and transformation.
• Familiarity with big data technologies and distributed computing frameworks (e.g., Hadoop, Spark).
• Understanding of data governance principles, data security, and regulatory compliance.
• Experience with data visualization and business intelligence tools (e.g., Tableau, Looker) is a plus
• Versatility: Experience across the entire spectrum of data engineering, including:
  • Cloud data stores (Example: Snowflake, GCP BigQuery, GCP BigTable, GCP FireStore, GCP CloudSQL)
  • Data pipeline and workflow orchestration tools (e.g., Dataflow, Airflow, Azkaban)
  • Data processing technologies
  • Data messaging technologies (e.g., GCP PubSub, Kafka)
• Strong Unix/Linux Shell scripting,
• Demonstrated knowledge of industry trends and standards
• Ability to think through multiple alternatives and select the best possible solutions for strategic and tactical business needs, and
• Excellent verbal and written communication

Nice to Have:

• Experience with GCP / AWS / Azure
• Machine Learning Engineering Certificate (with corresponding work experience)