Notebooks
This module provides a playground to the users who want to write their own code for ETL pipeline or any other purpose. In a notebook there can be multiple code blocks in which user can write his code and that can run/execute independently to solve a purpose. Following are the main features that are available under this module.
Write your own code.
Multiple languages support.
ETL pipeline (Stream/Batch).
Code Snippets for periodic execution.
Scheduler (CRON Jobs).
To create a Notebook go to Bigdata > Notebooks and click on Add New, fill in the basic details - Name & description then click on Next button.
To write your code snippets click on Add New CodeBlock. Configure the details - language, ETL, type, sample code; of code block according to your use case and proceed with the coding.
In this example, we will read data from MongoDB collection using Apache Spark SQL in Scala.
Click on save button after writing the code and click on *Run button available in actions of codeblock and then debug button to see the output of the code written.
Once you are satisfied with the code and its output then save the code block and save the notebook for future access.
Also you can schedule the execution of a code block. All you need to do is click on the Schedule button available in actions of code block then define the scheduler details - Hourly, Daily or Custom Cron expression, click on Save button.
Then to enable the configured scheduler click on green play button i.e. Schedule Job button in the top left side of code block. Once this is enabled then the code block will start to execute periodically according to defined scheduler.
Last updated
