Job Duties:

• Define and manage the data architecture framework, standards, and principles, including modeling, metadata, security, reference, and master data.
• Define reference architecture, and models to provide a common vocabulary, reusable components, and industry best practices.
• Create solution frameworks integrating large or complex data sets.
• Lead all data modeling efforts within DataBricks, including the design of data structures and the identification of business transformation logic.
• Analyzing and translating business requirements into conceptual and fully detailed logical data models.
• Creating logical data models based on existing applications and databases.
• Working with business architects, and data stewards to capture business requirements in a Logical Data Model.
• Transform the logical representation of the model into a physical representation and work with data engineering team to instantiate and manage the data.
• Contribute to assessment of appropriate data platform(s) for solutioning efforts.
• Collaborate with the BI and Analytics teams on creating the optimized, reusable semantic model, complete with metadata and lineage information.
• Define and govern data modeling and design standards, tools, best practices, and related development for enterprise data models
• Maintain the metadata - the "data about the data" describing the data model, its structure, and semantics
• Recommend data and design patterns for common business functionality
• Assist in multi-phase roadmaps within and across functional teams
• Provide insight and technology recommendations that result in enterprise solutions and architectural risk mitigation.
• Contribute to innovation strategies by exploring, investigating, recommending, benchmarking and implementing new data centric technologies for the platform.
• Identify, design, and implement internal process improvements such as automation of manual processes, optimization of data delivery and re-design of existing infrastructure for greater scalability.
• Build the infrastructure and processes required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and/or other data technologies.
• Create and maintain data-related documentation (e.g. data dictionaries, data maps, data models etc.)
• Work with data engineering and analytics experts to strive for greater functionality in our data systems and incorporation of industry best practices

Work experience / Technologies required for the position :

• Experience building and optimizing Big Data, Data pipelines, architectures and data sets.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Strong analytic skills related to working with unstructured datasets.
• Build processes supporting data transformation, data structures, meta data, dependency and workload management.
• A successful history of manipulating, processing and extracting value from large disconnected data sets.
• Working knowledge of message queuing, stream processing, and highly scalable Big Data, data stores.
• Experience supporting and working with cross-functional teams in a dynamic environment.
• Experience in a Data Engineer or similar roles, who has attained a Bachelor’s degree in Bachelor’s degree in Engineering, Computer Science, Information Technology, related field or equivalent work experience.
• Experience with big data tools: Hadoop, Spark, Kafka, etc.
• Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
• Experience with AWS cloud services: EC2, EMR, RDS, Redshift
• Experience with stream-processing systems: Storm, Spark-Streaming, etc.
• Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.