Instruct the Notebook

Overview

The Generate Notebook Instructions enables users to define what and how the AI Data Engineer should transform and process source data. Whether you provide instructions manually or let the AI derive them from documentation in a folder, this feature puts you in control of the configuration logic. At the same time, the AI does the heavy lifting.

The interface is divided into two main areas:

1. Resources – Where you define and attach your data sources.

2. Instructions – Where you specify the details of what to do with that data.

Once you provide these details, the AI Data Engineer will generate a notebook that automates your data workflow.

Auto-Configure Instructions

1. Select Generate Notebook

Generate Notebook Instructions

Resources Panel

Identify all the data inputs the AI Data Engineer will use.

Adding Resources:

1. Click Add resource.

2. Select files or define static paths (e.g., /erp/invoices).

3. Drag and drop resource paths into the panel for quicker setup.

4. Select Save.

All provided paths are interpreted and included automatically; no extra configuration is needed.

Resource Guidance

When using resources to generate instructions:

• Limit of 10 files per resource set

• All files must be relevant to the current data use case

• Accepted formats include DOCX, PDF, TXT, CSV, XLSX, SQL, and code files

1. Select Add resources

2. Select Resources

Instructions Panel

Define precise tasks for your AI Data Engineer. This is ideal when you have exact requirements or need custom, one-off logic.

Adding Instructions:

1. Populate the instructions section with the following structure below.

2. Select Generate

Structure:

• # Source: Identify what data is being transformed. This is the path or dataset where the data comes from. Example: /erp/invoices

• # Destination: Output path or location where the processed data will be saved. This often pre-populates from your Fabric workspace, but you can also provide instructions to create a table(s). Example: /analytics/monthly_summary

• # Instructions: Describe transformations, validation rules, mappings, business logic, or workflow in plain language. Example:

  Copy

  Filter for invoices from the past month,
  convert currency to USD,
  and aggregate totals by client.

These instructions act as guardrails for the AI, helping it:

• Stay aligned with business rules

• Ensure data integrity

• Avoid brittle or incorrect transformations

Notebook Best Practices

• Be Specific: Clear, precise instructions produce better results.

• Use Paths Consistently: Ensure your source/destination paths are valid and accessible.

• Describe Transformations in Order: List processing steps sequentially.

• Validate Output Requirements: Confirm the destination matches your data format needs.

Real-World Example

Let’s say your destination is a employee_pets table, and you want the AI to extract employee and pet information from messy spreadsheets. You could enter instructions such as the following:

The AI will parse that information and present it back to you as an editable instruction template. You can then adjust as needed.