Databricks : Certified-Data-Engineer-Professional Questions & Answers

Databricks Certified-Data-Engineer-Professional Exam Syllabus Topics:

Databricks Certified Data Engineer Professional Sample Questions:

1. Predictive Optimization is an automated Databricks service enabled by default for Unity Catalog Managed tables. It helps maintain Delta tables by continuously optimizing them to ensure optimal performance and costs. Which two operations does Predictive Optimization run to maintain the Delta tables? (Choose two.)

A) COMPACT
B) PARTITION BY
C) OPTIMIZE
D) BUCKETING
E) ANALYZE

2. A data engineer is using Lakeflow Declarative Pipeline to propagate row deletions from a source bronze table (user_bronze) to a target silver table (user_silver). The engineer wants deletions in user_bronze to automatically delete corresponding rows in user_silver during pipeline execution.
Which configuration ensures deletions in the bronze table are propagated to the silver table?

A) Use apply_changes without CDF and filter rows where _soft_deleted is true.
B) Enable Change Data Feed (CDF) on user_bronze, read its CDF stream, and use apply_changes() with apply_as_deletes=True for user_silver.
C) Enable CDF on user_silver, read its transaction log, and use MERGE to sync deletions.
D) Configure VACUUM on user_bronze to delete files, then rebuild user_silver from scratch.

3. A table in the Lakehouse named customer_churn_params is used in churn prediction by the machine learning team. The table contains information about customers derived from a number of upstream sources. Currently, the data engineering team populates this table nightly by overwriting the table with the current valid values derived from upstream data sources.
The churn prediction model used by the ML team is fairly stable in production. The team is only interested in making predictions on records that have changed in the past 24 hours.
Which approach would simplify the identification of these changed records?

A) Calculate the difference between the previous model predictions and the current customer_churn_params on a key identifying unique customers before making new predictions; only make predictions on those customers not in the previous predictions.
B) Modify the overwrite logic to include a field populated by calling
spark.sql.functions.current_timestamp() as data are being written; use this field to identify records written on a particular date.
C) Convert the batch job to a Structured Streaming job using the complete output mode; configure a Structured Streaming job to read from the customer_churn_params table and incrementally predict against the churn model.
D) Replace the current overwrite logic with a merge statement to modify only those records that have changed; write logic to make predictions on the changed records identified by the change data feed.
E) Apply the churn model to all rows in the customer_churn_params table, but implement logic to perform an upsert into the predictions table that ignores rows where predictions have not changed.

4. Spill occurs as a result of executing various wide transformations. However, diagnosing spill requires one to proactively look for key indicators.
Where in the Spark UI are two of the primary indicators that a partition is spilling to disk?

A) Driver's and Executor's log files
B) Query's detail screen and Job's detail screen
C) Stage's detail screen and Executor's log files
D) Stage's detail screen and Query's detail screen
E) Executor's detail screen and Executor's log files

5. The business reporting tem requires that data for their dashboards be updated every hour. The total processing time for the pipeline that extracts transforms and load the data for their pipeline runs in 10 minutes.
Assuming normal operating conditions, which configuration will meet their service-level agreement requirements with the lowest cost?

A) Schedule a job to execute the pipeline once an hour on a dedicated interactive cluster.
B) Schedule a Structured Streaming job with a trigger interval of 60 minutes.
C) Configure a job that executes every time new data lands in a given directory.
D) Schedule a job to execute the pipeline once an hour on a new job cluster.

Solutions: