How to use
================

.. toctree::
   :maxdepth: 3
   :caption: Contents:

Introduction
------------

This repo contains all the log processing tools centralized. This tools can use user generated protobuf mavlink content.
They are composed of cpp buildable files and python scripts. Shell and bat scripts are also provided to build the cpp files for windows
or unix platforms.

- The logging mechanism is described in the stream logger library: `libDM_stream_logger <../libdm_stream_logger/index.html>`_.
- The protobuf mechanism used is described in the protobuf library: `libDM_protobuf <../libdm_protobuf/index.html>`_.

The proposed logging mechanism allows unique file logging for:
- protobuf events
- mavlink events
- binary logs
- text logs
...

parse_binDM
-----------

This part of the repo contains the tools to parse the binary logs logged inside the protolog. As this kind of logs are often high frequency
with a lots of variables, the tools are optimized to be fast and efficient.
The main script used is located in `/parse_binDM/src/parse_log.cpp`

It can be recompiled with the provided build script located at `/parse_binDM/build_unix.sh` or `/parse_binDM/build_windows.bat`

The executable files are located in `/parse_binDM/builds/xxx`

The script can generates outputs in `csv`or `.mat` format. From a binary file.

It can be used with the following command line:

.. code-block:: bash

    ./parse_log_[platform] -i [input_file] -o [output_file] -f [format]

with :

.. hint::

    - input_file: input file path
    - output_file: output file path
    - output_format: mat or csv
    - debug: enable debug printf


parse_protolog
-----------------

A decoding script is provided to decode completely the protobuf logs, including, events, binary logs, mavlink messages, etc.
The script is located in the folder parse_protolog/tools/decode_protobuf.py.

The script can be used as follow:

- Install the requirements:

.. code-block:: bash

    pip install -r requirements.txt

- Run the script:

.. code-block:: bash

    python decode_protobuf_log.py --input_file log.binDM

Full options with python3 decode_protobuf_log.py -h:

.. code-block:: bash

    usage: python3 decode_protobuf_log.py --input_file ../test/log_test.binDM --format csv --do_zip True --parse_protolog_custom_binary decode_protobufDM_linux --parse_binary_custom_binary parse_log_linux

    Extract protobuf encoded logs

    options:
    -h, --help            show this help message and exit
    --input_file INPUT_FILE
                            Input file path
    --format FORMAT       Output format for binary data (mat or csv)
    --do_zip DO_ZIP       Zip the extracted file?
    --parse_protolog_custom_binary PARSE_PROTOLOG_CUSTOM_BINARY
                            Custom compiled protolog extractor
    --parse_binary_custom_binary PARSE_BINARY_CUSTOM_BINARY
                            Custom compiled binary log extractor

The script will create a new folder containing the raw extracted data and decoded files with the indicator **_decoded**.

.. note::

    If the protobuf/mavlink built files are not present or not up to date, they can be automatically imported with :

    .. code-block:: python

        python3 get_messaging.py

    The files will be imported either from the user workspace, directly taking the files from the protobuf/mavlink repos. If these folders are not present, the script will clone the current repos.

The extraction results should looks like this:

.. image:: images/command_extraction.png
    :height: 300px
    :alt: Protobuf decoded

The extracted data will be stored in the `output` folder:

.. image:: images/result_extraction.png
    :height: 300px
    :alt: Protobuf decoded

The content of the different files extracted should looks like:

- The binary log:
    .. image:: images/binary_extracted.png
        :height: 300px
        :alt: Protobuf decoded

- The protobuf events:
    .. image:: images/quaternion_extracted.png
        :height: 300px
        :alt: Protobuf events

- The mavlink messages:
    .. image:: images/mavlink_extracted.png
        :height: 300px
        :alt: Mavlink decoded

- The text logs:
    .. image:: images/ulog_extracted.png
        :height: 300px
        :alt: Text logs