Pass the Databricks ML Data Scientist Databricks-Machine-Learning-Professional Questions and answers with CertsForce

The mlflow.register_model operation can be used to add a new version of an MLflow Model to an existing MLflow Model Registry model. This operation takes the URI of the model source (such as an MLflow run or a local path) and the name of the registered model as arguments, and returns a ModelVersion object that contains metadata about the registered model version. The mlflow.register_model operation can also create a new registered model if the name does not exist in the Model Registry.

The other options are incorrect for the following reasons:

B. MlflowClient.update_registered_model is used to update the metadata of an existing registered model, such as the name or description, not to add a new version of the model.

C. mlflow.add_model_version does not exist as a valid MLflow operation.

D. MlflowClient.get_model_version is used to fetch the metadata of a specific version of a registered model, not to add a new version of the model.

E. The machine learning engineer does not need to create an entirely new MLflow Model Registry model, as they can use the mlflow.register_model operation to add a new version of the model to the existing “project” model.

References: Adding an MLflow Model to the Model Registry, mlflow.register_model

The log_metric function is used to log a numeric metric value to the current MLflow run. It takes the name of the metric and the value as arguments. In this case, the name of the metric is “rmse” and the value is the variable rmse. The log_metric function can be used multiple times to log different metrics or update the same metric. The log_artifact function is used to log a file or directory as an artifact to the current MLflow run. The log_model function is used to log a model with a specified flavor to the current MLflow run. The log_param function is used to log a key-value pair as a parameter to the current MLflow run. There is a way to store values like this using the log_metric function, so option E is incorrect.https://docs.databricks.com/

Structured Streaming is a scalable and fault-tolerant stream processing engine built on the Spark SQL engine. It allows users to express streaming computations the same way as batch computations on static data, using DataFrame and Dataset APIs. Structured Streaming can handle both unbounded and bounded data sources, and can process data in micro-batches or continuously. Structured Streaming can be used to provide continuous processing of data for machine learning applications, such as data ingestion, preprocessing, feature engineering, and model inference. StructuredStreaming can also integrate with MLflow and Delta Lake to enable end-to-end machine learning pipelines with tracking, reproducibility, and governance123 References:

Structured Streaming Programming Guide - Spark 3.2.0 Documentation

Structured Streaming + Kafka Integration Guide (Kafka broker version 0.10.0 or higher) - Spark 3.2.0 Documentation

Machine Learning with Structured Streaming - Databricks

[Continuous Machine Learning with Structured Streaming and MLflow - Databricks]

MLflow provides a module called mlflow.pyfunc that can be used to deploy any Python model as a PySpark UDF. The mlflow.pyfunc.spark_udf function takes a Spark session and a model URI, and returns a Spark UDF that can be applied to a Spark DataFrame. The model URI can be a local path, a remote path, or a run ID with a relative path within the MLflow tracking server. The mlflow.pyfunc.spark_udf function can be used with any model that has a predict function, such as scikit-learn, xgboost, etc12

The other code blocks are incorrect because:

A. The mlflow.pyfunc.spark_udf function requires a Spark session as the first argument, not a Spark DataFrame.

B. It is possible to deploy a scikit-learn model on a Spark DataFrame using mlflow.pyfunc.spark_udf or other methods, such as broadcasting the model or using Fugue.

C. There is no module called sklearn.spark_udf. To use a scikit-learn model as a Spark UDF, mlflow.pyfunc.spark_udf or other methods should be used.

D. There is no method called spark.spark_udf. To create a Spark UDF, the spark.udf.register or pyspark.sql.functions.udf methods should be used.

References:

mlflow.pyfunc — MLflow 2.9.1 documentation

Deploy models for batch inference and prediction - Databricks

Jensen-Shannon (JS) distance is a symmetric measure of the distance between two probability distributions. It is based on the Kullback-Leibler (KL) divergence, but unlike KL, it does not depend on which distribution is considered the reference and which is considered the comparison. JS distance is also bounded between 0 and 1, which makes it easier to interpret and compare across different features.

Kolmogorov-Smirnov (KS) test is a non-parametric test that compares the cumulative distribution functions (CDFs) of two samples. It calculates the maximum difference between the CDFs and compares it to a critical value to determine if the samples are from the same distribution. KS test is sensitive to the shape, location, and scale of the distributions, but it can also be too sensitive for large datasets, where small changes can result in significant differences.

JS distance is more robust than KS test for numeric feature drift detection because it can handle large datasets without being too sensitive to minor fluctuations. It can also capture changes in the shape and spread of the distributions, not just the location. JS distance does not require any manual threshold or cutoff determinations, unlike KS test, which requires choosing a significance level and a critical value. JS distance can also be generalized to multivariate cases, while KS test is limited to univariate cases. References:

Which test is the best? We compared 5 methods to detect data drift on large datasets

The Jensen-Shannon Divergence: A Measure of Distance Between Probability Distributions

Kullback–Leibler vs Kolmogorov-Smirnov distance

The client.delete_registered_model method of the MlflowClient class can be used to delete a model from the MLflow Model Registry. The method takes the following parameter:

name: The name of the registered model to delete.

The method deletes the registered model and all its versions from the Model Registry. The method does not return any value1

The other options are incorrect because:

A. The client.transition_model_version_stage method can be used to transition a model version to a different stage, such as None, Staging, Production, or Archived. The method does not delete the model or the model version from the Model Registry2

B. The client.delete_model_version method can be used to delete a specific model version from the Model Registry. The method does not delete the registered model or all its versions from the Model Registry3

C. The client.update_registered_model method can be used to update the metadata of a registered model, such as its name or description. The method does not delete the model or its versions from the Model Registry4

D. There is no method called client.delete_model. To delete a model from the Model Registry, the client.delete_registered_model method should be used.

References:

mlflow.tracking.client.MlflowClient.delete_registered_model — MLflow 2.9.1 documentation

mlflow.tracking.client.MlflowClient.transition_model_version_stage — MLflow 2.9.1 documentation

mlflow.tracking.client.MlflowClient.delete_model_version — MLflow 2.9.1 documentation

mlflow.tracking.client.MlflowClient.update_registered_model — MLflow 2.9.1 documentation

MLflow Model Serving is a feature that allows users to host MLflow models as REST endpoints on Databricks1. To query a model version that is in the Production stage of the MLflow Model Registry, the user can use the following model URI format: https:// /model-serving///invocations2. In this case, the model name is recommender and the stage is Production, so the correct model URI is https:// /model-serving/recommender/Production/invocations. The other options are incorrect because they either use the wrong prefix (model instead of model-serving), the wrong stage name (stage-production instead of Production), or require the version number of the model version in Production, which is not necessary for querying the model by stage. References: Announcing MLflow Model Serving on Databricks, [MLflow Model Serving], [MLflow Model Registry]

The correct line of code is “MODEL_VERSION_TRANSITIONED_STAGE”, which is the name of the event that occurs when a model version transitions to any stage in the MLflow Model Registry. This event can be used to trigger a webhook that will execute a Databricks Job based on the model version and the stage it transitions to. The other options are either invalid event names or too specific for the task.

References: MLflow Model Registry Webhooks, MLflow Model Registry Events

The Feature Store Client fs provides a method called create_table that can be used to create and populate a Feature Store table from a Spark DataFrame. The create_table method takes the following parameters:

name: The name of the feature table to create. It must include the database name, such as ‘recommender_system.new_table’.

primary_keys: The name or list of names of the columns that uniquely identify each row in the feature table. For example, ‘customer_id’ or [‘customer_id’, ‘date’].

df: The Spark DataFrame that contains the data to populate the feature table. It must have the same schema as the feature table, and include the primary key columns.

description: An optional string that describes the feature table.

partition_columns: An optional list of column names to partition the feature table by. For example, [‘date’, ‘region’].

online_store: An optional boolean flag that indicates whether to publish the feature table to an online store. The default value is False.

online_store_options: An optional dictionary that specifies the configuration options for the online store, such as table name, database name, and write mode. For example, {‘table_name’: ‘new_table_online’, ‘database_name’: ‘recommender_system’, ‘write_mode’: ‘overwrite’}.

The create_table method creates a Delta table with the specified name, primary keys, partition columns, and description, and writes the data from the DataFrame to the table. If online_store is True, it also publishes the feature table to an online store with the specified options. The create_table method returns a FeatureTable object that represents the feature table12

The other code blocks are incorrect because:

B. The create_table method requires the df parameter to populate the feature table with data.

C. The write method of the DataFrame does not have a mode called ‘fs’ or a path parameter. To write a DataFrame to a Delta table, the mode should be one of ‘append’, ‘overwrite’, ‘ignore’, or ‘error’, and the format should be ‘delta’. To write a DataFrame to a Feature Store table, the create_table or write_table method of the Feature Store Client should be used.

D. The create_table method does not have a function parameter. To use a Python function to compute the features, the function should be called first and the output DataFrame should be passed to the df parameter.

E. The write method of the DataFrame does not have a mode called ‘feature’ or a path parameter. To write a DataFrame to a Feature Store table, the create_table or write_table method of the Feature Store Client should be used.

References:

Feature Store Python API reference - Databricks

Work with features in Workspace Feature Store - Databricks

An HTTP Webhook is a mechanism that allows you to register a callback that is triggered by an event, such as a model registry event. The callback is an HTTP request that is sent to a specified URL, which can invoke an action or a notification on another platform or service. An HTTP Webhook is required for use cases that involve integrating the model registry with external tools or workflows that are not supported by Databricks1.

Sending a message to a Slack channel when a model version transitions stages is a use case that requires the use of an HTTP Webhook. This is because Slack is an external platform that is not natively integrated with Databricks, and the model registry events are not directly accessible by Slack. Therefore, to send a message to a Slack channel, you need to register an HTTP Webhook that is triggered by the model registry event of interest, such as MODEL_VERSION_TRANSITIONED_STAGE. The HTTP Webhook then sends a request to the Slack API endpoint that corresponds to the channel and the message content2.

The other options are incorrect because:

Option A: Starting a testing job when a new model is registered does not require the use of an HTTP Webhook, but rather a job registry webhook. A job registry webhook is a type of webhook that triggers a job in a Databricks workspace when a model registry event occurs. A job registry webhook can be created using the Databricks REST API or the Python client databricks-registry-webhooks on PyPI3.

Option B: Updating data in a source table for a Databricks SQL dashboard when a model version transitions to the Production stage does not require the use of an HTTP Webhook, but rather a Databricks SQL trigger. A Databricks SQL trigger is a mechanism that allows you to execute a SQL query or a notebook when a specified condition is met, such as a time interval or a file arrival. A Databricks SQL trigger can be created using the Databricks SQL UI or the Databricks REST API4.

Option C: Sending an email alert when an automated testing job fails does not require the use of an HTTP Webhook, but rather a job alert. A job alert is a feature that allows you to send an email notification when a job run meets a specifiedcondition, such as a failure, a timeout, or a success. A job alert can be created using the Jobs UI or the Databricks REST API5.

Option D: None of these use cases require the use of an HTTP Webhook is incorrect, as option E does require the use of an HTTP Webhook. References: MLflow Model Registry Webhooks on Databricks, Streamline MLOps With MLflow Model Registry Webhooks, Databricks SQL Triggers, Job Alerts, [Slack API]