Creating a New Jupyter Notebook

Create a new Jupyter notebook (.ipynb) in JupyterHub to develop code for visualizing, analyzing, and processing data from your tests, measurements, assets, and more.

Before you create a Jupyter notebook, complete the following tasks:
  • Install Jupyter Notebooks for SystemLink from NI Package Manager to access predefined Jupyter notebooks that leverage SystemLink Python APIs.
  • Append /niapis/python/index.html to your the SystemLink web application URL to access SystemLink's Python APIs.
  • Learn about the user interface and structure of a Jupyter notebook.
  • Install Python libraries you want to use to enhance your data analysis and processing. NI recommends using PyPI to obtain packages.
  • Learn about the built-in magic commands the IPython kernel enables.
Complete the following steps to create a Jupyter notebook.
  1. In the SystemLink web application, click Jupyter.
  2. Under Notebook, click Python 3.
    A Jupyter notebook launches and starts a kernel to run Python in the notebook.
  3. Rename the notebook to describe the type of report it will execute.
  4. In a code cell, import the Python modules, libraries, and widgets you need to build your notebook and make it interactive.
    For example, if you want to import Pandas to build and handle dataframes and Scrapbook to execute the notebook and capture results, you need to implement the following code:
    import copy
import datetime
import dateutil.parser
import pandas as pd
import scrapbook as sb
from dateutil import tz
  5. Use one of the SystemLink Python APIs to access the data you want to visualize, analyze, or process.
    For example, if you want to access data from the SystemLink Test Monitor service, your code would look like the following command:
    from systemlink.clientconfig import get_configuration
from systemlink.clients.nitestmonitor import *
  6. Establish a connection to your SystemLink Server.
    For example, if you want to connect a Test Monitor client to the server, use the following command: 
    http_client_config = get_configuration(route_name='nitestmonitor')
  7. Define the parameters and metadata for the notebook.
    1. In a code cell, define the parameters.
      For example, if you wanted to filter your test results to a specific time range and group them by the day your test system acquired them, your code may look like the following:
      filter = 'startedWithin <= "30.0:0:0"'
group_by = 'Day'
    2. On the right sidebar, open the Property Inspector pane and add the parameters, their default values, and outputs to the Cell Metadata code block.
      For example, your code may look like the following:
      "papermill": {
        "parameters": {
            "group_by": "Day",
            "results_filter": "startedWithin <= \"30.0:0:0\"",
        }
    },
    "systemlink": {
        "namespaces": [
            "ni-testmanagement"
        ],
        "outputs": [
            {
                "display_name": "This will show in dashboard output selector",
                "id": "data_frame_output",
                "type": "data_frame"
            },
            {
                "display_name": "This will show in dashboard output selector",
                "id": "scalar_output",
                "type": "scalar"
            }
        ],
        "parameters": [
            {
                "display_name": "Group By",
                "id": "group_by",
                "options": [
                    "Day",
                    "System",
                    "Test Program",
                    "Operator",
                    "Product"
                ],
                "type": "string"
            },
            {
                "default_display": {
                    "startedWithin": {
                        "unit": "DAYS",
                        "value": 30
                    }
                },
                "display_name": "Results Filter",
                "id": "results_filter",
                "type": "test_monitor_result_query"
            },
        ],
        "version": 2
    },
    "tags": [
        "parameters"
    ]
}
    3. For namespaces, enter one or more of the following namespaces depending on where you want to view your report:
      Namespace Report display location
      ni-assetmanager Asset Manager
      ni-testmanagement Test Monitor
      Any namespace you define Any other client that uses namespaces to query notebooks
    4. For version, enter one of the following version numbers depending on how you want to use your notebook:
      Goal Version
      • You want to view your data in Test Insights under Reports.
      • You are customizing a notebook installed with a previous version of SystemLink.
      		 1
      • You want to return multiple outputs with one notebook.
      • You want to use a scalar output type.
      • You want to see your data in a dashboard.
      		 2
  8. Query a SystemLink data service for the data you want visualize, analyze, or process.
    For example, if you want to query the Test Monitor service for test results in ascending order, your code may look like the following: 
    results_api = ResultsApi(api_client=ApiClient(http_client_config))
query = ResultsAdvancedQuery(filter=filter, order_by=[ResultSortDefinitionObject(field=ResultField.STARTED_AT)])
query_response = await results_api.query_results_v2(post_body=query)
results = query_response.results

results_list = [result.to_dict() for result in results]
  9. Format the data in a Pandas dataframe based on how you want to group the data returned from the query.
    For example, if you want to group your test results by their status, your code may look like the following:
    group_names = []
for result in results_list:
    if grouping in result:
        group_names.append(result[grouping])

formatted_results = {
    'id': [result['id'] for result in results_list],
    'status': [result['status']['status_type'] for result in results_list],
    grouping: group_names
}

df_results = pd.DataFrame.from_dict(formatted_results)
  10. Configure how you want to visualize, analyze, or process the data.
    The following examples are some of the ways you can configure the data.
    • Filter out results you do not want to include in the report.
    • Convert timestamps.
    • Compute data within a timeframe.
    • Group data in a specific way.
  11. Convert the Pandas dataframe into the SystemLink reports output format.
    For example, the code may look like the following:
    result.append({
    'type': 'data_frame',
    'id': 'data_frame_output', 
    data': [{
        'format': 'XY',
        'x': ['2018-11-17T00:00:00', '2018-11-18T00:00:00', ...],
        'y': [94.0, 89.9, ...]
    }],
    'config': {
        'title': 'Title',
        'graph': {
            'axis_labels': ['x-axis-label', 'y-axis-label'],
            'tick_labels': [{'x': 0, 'label': 'tick label 0', ... }],
            'orientation': 'VERTICAL',
            'plot_style': ['SCATTER'],
            'plot_color': ['blue']
        }
    }
})
    Tip To ensure Test Monitor supports the dataframe output, verify the value of the id (data_frame_output) is correct in the cell and in the Code Metadata for the parameter code cell. Refer to step 7b for more information.
    Note If you use the V2 report format in a Jupyter notebook, you can only visualize the report results on a dashboard.
  12. Add a new scalar output.
    Note Refer to the SystemLink examples repository in GitHub for more information on adding a scalar output.
    For example, your code may look like the following: 
    result.append({
    'type': 'scalar',
    'id': 'scalar_output',
    'config': {
        'title': 'Scalar Output Title'
    },
    'value': 3
})
  13. Record results with Scrapbook.
    Refer to the following code example for optimal parsing in the SystemLink web application:
    sb.glue('result', result)
  14. Add a new cell in the notebook.
  15. On the toolbar, choose Markdown from the dropdown menu to add documentation about your code to the notebook.
  16. On the menu bar, select Run » All Cells to verify the notebook returns results or processes data as expected.
  17. If the notebook returns an error, add import pdb; pdb.set_trace() to the code cell with the error to use the built-in Python debugger.
  18. Click Save or use the Ctrl + S shortcut.
  19. Optional: To run your notebook at a scheduled time, create an Analysis Automation procedure and add a scheduled task for the procedure.
After you create your notebook, you can bind it as a data source to a tile and monitor the results on a dashboard. If you need to share your notebook, refer to Sharing a Jupyter Notebook for more information.