Pass the Databricks ML Data Scientist Databricks-Machine-Learning-Professional Questions and answers with CertsForce

To view all of the active MLflow Model Registry Webhooks for a specific model, the machine learning engineer needs to replace POST with GET in the call to http request. This is because the HTTP method for listing webhooks is GET, not POST. The GET method is used to retrieve information from the server, while the POST method is used to send data to the server1. The syntax of the http request function is as follows:

Python

http_request(host_creds, endpoint, method, json=None, headers=None)

AI-generated code. Review and use carefully. More info on FAQ.

The other options are incorrect because:

Option A: There are no necessary changes is incorrect, as the code block will not work with the POST method.

Option B: Replace list with view in the endpoint URL is incorrect, as the endpoint URL for listing webhooks is “/api/2.0/mlflow/registry/webhooks/list?model_name=model”, not “/api/2.0/mlflow/registry/webhooks/view?model_name=model”. There is no view endpoint for webhooks2.

Option D: Replace list with webhooks in the endpoint URL is incorrect, as the endpoint URL for listing webhooks is “/api/2.0/mlflow/registry/webhooks/list?model_name=model”, not “/api/2.0/mlflow/registry/webhooks/webhooks?model_name=model”. The list keyword is required to specify the list operation2.

Option E: Replace POST with PUT in the call to http request is incorrect, as the HTTP method for listing webhooks is GET, not PUT. The PUT method is used to update data on the server, not to retrieve information1. References: HTTP request methods, List webhooks

MLflow provides a module called mlflow.shap that can be used to automatically calculate and log a Shapley feature importance plot using the SHAP library. The mlflow.shap.log_explanation function takes a predict function and a sample of input data, and computes the SHAP values and base values for each feature. It then logs the following artifacts to the current run:

Base values

SHAP values (computed using shap.KernelExplainer)

Summary bar plot (shows the average impact of each feature on model output) The mlflow.shap.log_explanation function can be used with any model that has a predict function, such as scikit-learn, xgboost, etc1234 References:

mlflow.shap — MLflow 2.9.1 documentation

Extract feature importance from a mlflow 1.9 PyFuncModel model

9.6 SHAP (SHapley Additive exPlanations) | Interpretable Machine Learning

load MLFlow model and plot feature importance with feature names

 In a continuous integration, continuous deployment (CI/CD) process for machine learning pipelines, the arrival of a new model version in the MLflow Model Registry is a common event that triggers the execution of automated testing. This is because the MLflow Model Registry is a centralized repository for managing the full lifecycle of MLflow models. It provides model lineage, model versioning, stage transitions, and annotations. When a new model version is registered, it can be automatically tested for quality, performance, and compliance using various tools and frameworks, such as MLflow Projects, MLflow Models, MLflow Tracking, and MLflow Model Serving. The automated testing can help ensure that the new model version meets the desired criteria and standards before it is deployed to production or other stages. The automated testing can also help detect and prevent any errors, bugs, or drifts that may affect the model functionality or accuracy. References:

MLflow Model Registry Documentation, p. 1-2

MLOps: Continuous delivery and automation pipelines in machine learning, Continuous integration and delivery (CI/CD) for ML, p. 5-6

Machine Learning Engineering with MLflow, Testing and Validating ML Models, p. 1-2

CI/CD for Machine Learning: What it is & Benefits in 2023 - AIMultiple, CI/CD for Machine Learning, p. 1-2

The fs.read_table operation can be used to return a Spark DataFrame of a data set associated with a Feature Store table. This operation takes the name of the Feature Store table and an optional time travel specification as arguments. The fs.create_table operation is used to create a new Feature Store table from a Spark DataFrame. The fs.write_table operation is used to write data to an existing Feature Store table. The fs.get_table operation is used to get the metadata of a Feature Store table, not the data itself. There is a way to accomplish this task with fs, so option D is incorrect. References:

Feature Store Client

Feature Store Tables

Concept drift in machine learning and data mining refers to the change in the relationships between input and output data in the underlying problem over time. In other domains, this change maybe called “covariate shift,” “dataset shift,” or “nonstationarity.” Concept drift can affect the performance and accuracy of predictive models that assume a static relationship between input and output variables. Concept drift can be caused by various factors, such as changes in user behavior, environmental conditions, market trends, etc. Concept drift can be detected and handled by various methods, such as periodic retraining, online learning, ensemble methods, etc References:

Concept drift - Wikipedia

A Gentle Introduction to Concept Drift in Machine Learning

Model Drift & Machine Learning: Concept Drift, Feature Drift, Etc.

Data Drift vs. Concept Drift: What Is the Difference?

Feature drift is a change in the distribution of the input data over time, which can affect the performance and accuracy of the model. Monitoring feature drift is important to ensure that the model is still valid and reliable for the current data. One simple and low-cost method of monitoring numeric feature drift is to track the summary statistics trends of the input features, such as mean, standard deviation, minimum, maximum, etc. These statistics can be computed easily and efficiently, and can provide a quick overview of the changes in the data distribution. If the summary statistics deviate significantly from the baseline values, it may indicate a feature drift. However, summary statistics trends may not capture all the nuances of the data distribution, such as outliers, multimodality, skewness, etc. Therefore, other methods, such as statistical tests or visualizations, may be needed to complement the summary statistics trends and provide a more comprehensive analysis of the feature drift123 References:

Monitoring Feature Drift - Databricks

Monitor feature attribution skew and drift | Vertex AI | Google Cloud

Monitoring Model Drift - 6 Different Methodologies - Qualdo

To compute predictions for spark_df when the missing feature values can be found in the Feature Store by searching for features by customer_id, you can use the following code block:

Python

# Get the missing features from the Feature Store using the model URI and the customer_id column

df = fs.get_missing_features(spark_df, model_uri, lookup_key="customer_id")

# Score the DataFrame using the model URI and the Feature Store Client

fs.score_batch(model_uri, df)

AI-generated code. Review and use carefully. More info on FAQ.

The fs.get_missing_features method takes a Spark DataFrame, a model URI, and a lookup key as arguments. It returns a new Spark DataFrame that contains the originalcolumns plus the missing features that are required by the model. The missing features are retrieved from the Feature Store by joining the DataFrame with the feature tables using the lookup key. The lookup key must match the primary key of the feature tables. The model URI must point to a registered model that was trained using features from the Feature Store1.

The fs.score_batch method takes a model URI and a Spark DataFrame as arguments. It applies the model to the DataFrame and returns a new Spark DataFrame that contains the original columns plus a prediction column. The model URI must point to a registered model that was trained using features from the Feature Store2.

The other options are incorrect because:

Option A: fs.score_model is not a valid method name, as it is missing an underscore. The correct method name is fs.score_batch2.

Option B: fs.score_model without getting the missing features will not work, as the model expects the DataFrame to have all the features that were used for training. The correct way is to use fs.get_missing_features before fs.score_batch12.

Option D: fs.score_batch without getting the missing features will not work, as the model expects the DataFrame to have all the features that were used for training. The correct way is to use fs.get_missing_features before fs.score_batch12.

Option E: fs.score_batch without specifying the lookup key will not work, as the fs.get_missing_features method requires a lookup key to join the DataFrame with the feature tables. The correct way is to use fs.get_missing_features with the lookup key “customer_id” before fs.score_batch12. References: Get missing features, Score batch

The Model Registry is a centralized model store that allows you to manage the full lifecycle of MLflow Models. It provides model lineage, versioning, stage transitions, and annotations. You can use the Model Registry to discover and share models, collaborate on moving them from experimentation to production, integrate with approval and governance workflows, and monitor model deployments and performance1. The other options are incorrect because:

Option A: Models is a component of MLflow that allows you to manage and deploy models from a variety of ML libraries to a variety of model serving and inference platforms. Models is not a centralized model store, but rather a convention for packaging machine learning models that offers self-contained code, environments, and models2.

Option C: Model Serving is a feature of Databricks that allows you to host MLflow Models as REST endpoints. Model Serving is not a centralized model store, but rather a model deployment option that enables low-latency inference for real-time applications3.

Option D: Feature Store is a feature of Databricks that allows you to store, discover, and share features for machine learning. Feature Store is not a centralized model store, but rather a centralized feature store that enables feature reuse, consistency, and governance across ML projects4.

Option E: Experiments is a component of MLflow that allows you to track experiments to record and compare parameters and results. Experiments is not a centralized model store, but rather a logging service that stores runs, metrics,parameters, tags, and artifacts for each experiment. References: MLflow Model Registry, MLflow Models, MLflow Model Serving, Databricks Feature Store, [MLflow Tracking]