Data Management#
After executing a program, its results can be exported from the database to a portable
HDF5 file with the to_hdf5() method, as shown
in Running a program on hardware.
In this tutorial, we go over of the structure of these
files.
[2]:
import h5py
import keysight.qcs as qcs
import numpy as np
file = h5py.File("swept_program.hdf5")
Accessing the program and data through QCS#
The program that was executed can be loaded from the file by calling
load().
[3]:
program = qcs.load("swept_program.hdf5")
We can inspect the program that was executed.
[4]:
program.draw()
|
Program
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
We can load the data. This will load from the hdf5 file, rather than the database.
[5]:
program.get_trace_array()
[5]:
<xarray.DataArray (channel: 4, sweep1: 11, shot: 10, sample: 384)> Size: 1MB
array([[[[-0.00373535, -0.00285645, -0.00373535, ..., -0.00153809,
-0.00175781, 0. ],
[-0.00219727, -0.00307617, -0.00439453, ..., -0.00175781,
-0.00065918, -0.00065918],
[-0.00263672, -0.00109863, -0.00219727, ..., -0.00087891,
-0.00021973, 0.00021973],
...,
[-0.00285645, -0.0032959 , -0.00307617, ..., -0.00197754,
-0.00109863, -0.00065918],
[-0.00373535, -0.00153809, -0.00109863, ..., -0.00043945,
-0.00065918, 0.00087891],
[-0.0032959 , -0.00153809, -0.00043945, ..., -0.00043945,
-0.00285645, -0.00175781]],
[[ 0.00109863, -0.00087891, -0.00263672, ..., -0.00351562,
-0.00351562, -0.00263672],
[-0.00131836, -0.00109863, -0.00241699, ..., -0.00439453,
-0.00351562, -0.00197754],
[-0.00307617, -0.00219727, -0.00109863, ..., -0.00351562,
-0.0032959 , -0.00439453],
...
[-0.00219727, 0.00021973, -0.00263672, ..., -0.00197754,
-0.00153809, -0.00307617],
[-0.00153809, -0.00153809, -0.00175781, ..., -0.00087891,
-0.00153809, -0.00131836],
[-0.00241699, -0.00263672, -0.00219727, ..., -0.00153809,
-0.00241699, -0.00175781]],
[[-0.00219727, -0.00285645, -0.00021973, ..., -0.00241699,
-0.00153809, -0.00131836],
[-0.00175781, -0.00395508, -0.00263672, ..., -0.00065918,
-0.00087891, -0.00109863],
[-0.00285645, -0.00175781, -0.00175781, ..., -0.00065918,
-0.00131836, -0.00043945],
...,
[-0.00395508, -0.00087891, -0.00065918, ..., -0.00153809,
-0.00175781, -0.00285645],
[-0.00219727, -0.00087891, -0.00065918, ..., -0.00175781,
-0.00175781, -0.00307617],
[-0.00175781, -0.00087891, -0.00087891, ..., -0.00153809,
-0.00263672, -0.00065918]]]])
Coordinates:
* channel (channel) <U6 96B 'digs_0' 'digs_1' 'digs_2' 'digs_3'
rf_0 (sweep1) float64 88B 3.85e+09 3.89e+09 ... 4.21e+09 4.25e+09
rf_1 (sweep1) float64 88B 3.9e+09 3.94e+09 3.98e+09 ... 4.26e+09 4.3e+09
rf_2 (sweep1) float64 88B 3.95e+09 3.99e+09 ... 4.31e+09 4.35e+09
rf_3 (sweep1) float64 88B 4e+09 4.04e+09 4.08e+09 ... 4.36e+09 4.4e+09
* shot (shot) int64 80B 0 1 2 3 4 5 6 7 8 9
* sample (sample) int64 3kB 0 1 2 3 4 5 6 7 ... 377 378 379 380 381 382 383
Dimensions without coordinates: sweep1Inspecting the file directly#
We can also use the hdf5 python library to inspect the file directly.
The attributes of the HDF5 are accessed with the attrs property.
[6]:
list(file.attrs.keys())
[6]:
['Program', 'ChannelMapper', 'FPGAPostprocessing', 'Shape', 'Version']
The FPGAPostprocessing attribute specifies whether these results are obtained with
(True) or without (False) hardware demodulation. The file contains either
IQ or trace data, respectively. This file contains trace data.
[7]:
file.attrs["FPGAPostprocessing"]
[7]:
False
The Shape attribute describes how the program was repeated during execution and is
similar to the shape of a program’s
repetitions. As we flatten the data, this
value describes how the data should be reshaped. For trace data the innermost value is
set to \(-1\) to account for different numbers of samples, following the
convention of, e.g., numpy.reshape().
[8]:
file.attrs["Shape"]
[8]:
array([[11, 10, -1]], dtype=int32)
The above value indicates that the trace data should be interpreted as an array of shape \((11, 10, n / (11 * 10))\), where \(n\) is the total number of data points.
Finally, the Program and the ChannelMapper used during its execution are
serialized and stored in the file.
Datasets on the file#
The file structure is different dependent on whether the program contains IQ or trace data.
Trace data#
An HDF5 file containing trace data will store each repeated acquisition in its own
dataset. The datasets are stored in groups named
DutChannel_{channel_number}_Acquisition_{acquisition_number} that are keys of the
file, each of which corresponds to a physical channel.
[9]:
list(file.keys())
[9]:
['DutChannel_1_Acquisition_0']
This file contains a single group for the acquisitions on DutChannel_20. The group
has a dataset with no additional attributes named trace.
[10]:
acquisition = file["DutChannel_1_Acquisition_0"]["trace"]
acquisition.shape
[10]:
(42240,)
We then reshape to and display the innermost value, the number of samples per repetition.
[11]:
np.reshape(acquisition, file.attrs["Shape"][0]).shape[-1]
[11]:
384
IQ data#
We now load a file that contains results from the same program run with hardware demodulation.
[12]:
file_demod = h5py.File("swept_program_demod.hdf5")
file_demod.attrs["FPGAPostprocessing"]
[12]:
True
Instead of using physical channels as keys, the IQ data uses virtual channels
to label the groups as
Channel_{channel_name}_{channel_label}_Acquisition_{acquisition_number}.
[13]:
list(file_demod.keys())
[13]:
['Channel_digs_0_Acquisition_0',
'Channel_digs_1_Acquisition_0',
'Channel_digs_2_Acquisition_0',
'Channel_digs_3_Acquisition_0']
The IQ data is separated into real and imaginary parts.
[14]:
acquisition_demod = file_demod["Channel_digs_0_Acquisition_0"]
print(acquisition_demod["iq_real"])
print(acquisition_demod["iq_imaginary"])
<HDF5 dataset "iq_real": shape (110,), type "<f8">
<HDF5 dataset "iq_imaginary": shape (110,), type "<f8">
The shape of the IQ data is exactly the program’s repetitions’ shape.
[15]:
np.reshape(acquisition_demod["iq_real"], file_demod.attrs["Shape"][0]).shape
[15]:
(11, 10)