Changelog

1.0.0-rc.44

This is an UNSTABLE release.

• jfjoch_broker: more general definition of scan result export

Braking changes:

• It removes additions to OpenAPI from 1.0.0-rc.43

• It makes changes to the “unit_cell” definition in OpenAPI specs. It might be harmless in some languages and may result in errors in other implementations.

1.0.0-rc.43

This is an UNSTABLE release.

• jfjoch_broker: Export grid scan results into a single data structure

1.0.0-rc.42

This is an UNSTABLE release.

• jfjoch_broker: Add pixel_sum to CBOR output.

• jfjoch_broker: Changes to sigma estimation in QuickIntegrate routine

• jfjoch_writer: Save pixel_sum

1.0.0-rc.41

This is an UNSTABLE release. This release includes multiple new features, it should not be used in production at the moment.

• jfjoch_broker: Estimate B-factor, mosaicity to evaluate crystal diffraction

• jfjoch_broker: Export GPU count via OpenAPI

• jfjoch_broker: Enable 2D azimuthal integration and PONI rotations for detector

• FPGA: Increase the number of integration bins to 2048

1.0.0-rc.40

This is an UNSTABLE release. This release includes multiple new features, it should not be used in production at the moment.

• jfjoch_broker: Jungfraujoch supports grid scan metadata, including dedicated plotting schemes and NXmx structures

• jfjoch_broker: Improve metadata for rotation data collection

• jfjoch_broker: Better handling of plotting

• jfjoch_broker: FFT based indexing

• jfjoch_broker: Integration, first try, results not saved at the moment

• jfjoch_broker: Internal improvements in image handling

• jfjoch_writer: Multiple adjustments adapt to changes in this release for new features

• jfjoch_writer: New state management model to improve clarity of error reporting

• jfjoch_viewer: Remote control via DBus

• Frontend: Multiple adjustments for new features

• Frontend: Grid scan plots

WARNING! OpenAPI contains breaking changes in regard to plotting results, so care has to be taken.

1.0.0-rc.39

• FPGA: Bugfix for pixel masked for data analysis if summation was on

• jfjoch_viewer: Fix segmentation fault when cursor was outside of image

1.0.0-rc.38

• jfjoch_broker: Neural net model is not linked with C++ code due to deployment issues, it is rather distributed as python code, connected via RES

• jfjoch_broker: Neural net model can use all 4 quadrants of the detector

• jfjoch_broker: For EIGER image time can be provided through /start

• jfjoch_viewer: Add image list option

• jfjoch_viewer: Drawing circular ROIs with shift

• jfjoch_viewer: Enable image summation

• jfjoch_viewer: Image reader is significantly reworked, hopefully without affecting the viewer

1.0.0-rc.37

• jfjoch_broker: Make locking rules more flexible

• jfjoch_broker: Load mask via SIMPLON interface for DECTRIS detectors

• jfjoch_viewer: Add status bar

1.0.0-rc.36

This is UNSTABLE release. Wait for new version to use in a production environment.

• jfjoch_broker: Support for Jungfraujoch Lite is enabled - software-based receiver for DECTRIS detectors (required a lot of refactoring, potentially leading to unstable code)

• jfjoch_broker: Enable Resonet support (ML-based diffraction resolution estimation)

• jfjoch_broker: Fix error in compression, where bitshuffle/LZ4 and bitshuffle/Zstd HDF5 headers were wrongly generated for 8-bit and 32-bit data

• jfjoch_writer: Increase buffering to 1000 images in the receiver

• jfjoch_writer: Images can be written as CBF or TIFF in addition to HDF5

1.0.0-rc.35

This is UNSTABLE release, not properly tested. Wait for new version for using production.

• jfjoch_broker: If module is delayed by more than 50 frames versus other modules, it will be ignored and receiver is not waiting.

• jfjoch_writer: Save EIGER energy threshold

• jfjoch_writer: Add /entry/sample/goniometer for compatibility with eiger2cbf program

1.0.0-rc.34

This is UNSTABLE release - introducing new features, but not properly tested. Wait for new version for using production.

• jfjoch_broker: More consistency for file format definition (breaking change in API from 1.0.0-rc.31 for file writer settings)

• jfjoch_broker: For storage cells mask is logical sum of detector bad pixels for all storage cells

• jfjoch_broker: Handle situation when detector doesn’t want to gracefully stop (to be tested)

• jfjoch_broker: Center-of-mass position and mean for ROI is added to available plots

• jfjoch_viewer: Can extract data analysis results from “legacy” format

• jfjoch_viewer: Display dataset name

• FPGA: Pixel mask is used for data analysis part even if it is not applied to pixels

• FPGA: Add pixel sum to module statistics

• FPGA: ROI number is reduced to 16, but pixel can belong to every defined ROI

• FPGA: Spot finder is back to full dynamic range (24-bit)

• FPGA: More debug features for internal FIFOs

Known issues:

• ROI count flag was added to firmware. For the time being the flag will be wrongly set to 10 due to mismatch of FPGA build scripts.

• EIGER data acquisition has an issue that is currently debugged

1.0.0-rc.33

• jfjoch_broker: Fix issue with EIGER settings being loaded improperly

1.0.0-rc.32

• jfjoch_broker: Refactor code for azimuthal integration for further improvements

• jfjoch_broker: Minor fix for EIGER (trim energies are manually set for E9M, to be fixed properly later)

• jfjoch_writer: Fix too much verbose information

• FPGA: Minor fixes to spot finder (enable two-pass operation and limit number range to int20)

1.0.0-rc.31

This is UNSTABLE release - introducing many features, but still needs more testing. Expecting soon to put bugfix release.

• jfjoch_writer: Allow to enable overwriting existing files (not enabled by default)

• jfjoch_writer: Add new HDF5 master file format, which uses HDF5 virtual data sets and links processing results to data files (not enabled by default)

• jfjoch_viewer: Image viewer work early test version

• jfjoch_broker: Fixes to counting packets per dataset/image

• jfjoch_broker: Image buffer is accessible for outside to check images

• jfjoch_broker: error/saturated pixels and dedicated ROI “beam” can be tracked online

• jfjoch_broker: Fix bug in handling pedestal G1/G2 count time for JUNGFRAU

• jfjoch_broker: Fix bug in applying pixel mask interfering with pedestal calculation

• jfjoch_broker: Fix bug in EIGER initializing

• jfjoch_broker: Save maximum pixel value to HDF5 file and export as Web plot

• PCIe driver: Add PCIe link speed and width

• FPGA: Improve counting error/saturated/min/max pixels

• FPGA: Spot finder is gradual column-wise (15 columns up/down) and fixed row-wise (32 pixel boxes); previously it was fixed both column- and row-wise with 32x32 pixel areas

• FPGA: Require Vivado 2022.2

Warning: There are breaking changes to HDF5 file format, renaming entries regarding image storage cell number and image collection efficiency.

1.0.0-rc.30

• jfjoch_writer: replace non-blocking with blocking operation on internal queues - less likely to “loose” images within the writer

1.0.0-rc.29

• jfjoch_broker: refactor logic regarding frame time and count time for more flexibility for EIGER and JUNGFRAU

• jfjoch_broker: readout time for EIGER is 3 us and JUNGFRAU is 20 us, this can be changed in input file

• jfjoch_broker: OpenAPI interface includes more ways to provide information on the status (error/warning/info)

• jfjoch_broker: ROIs handling via OpenAPI and frontend is more user friendly

Warning - two breaking changes to OpenAPI:

• Handling of ROIs is through /config/roi path only for both circle and box ROIs, path in /roi are no longer accessible

• broker_status structure introduced in 1.0.0-rc.28 has member message and not error_message to allow handling info/warning messages as well

1.0.0-rc.28

• jfjoch_broker: save error message for initialization and data collection and provide these with OpenAPI

• jfjoch_broker: fixed issue when in error state, response to /wait_till_done was not complaint to OpenAPI specs

• jfjoch_test: remove header that failed when CUDA is absent during compilation

• frontend: add soft trigger button in data collection tab

• frontend: show error message when in error state

• CMake: add option to force compilation without CUDA (-DJFJOCH_USE_CUDA=OFF)

1.0.0-rc.27

• jfjoch_broker: add option to select electron source in instrument metadata, adapt wavelength calculation

• jfjoch_broker: update pistache web server version

• jfjoch_writer: minor changes to republish logic

• Improvements to documentation

1.0.0-rc.26

• jfjoch_broker: implement ZeroMQ stream for image metadata information

• jfjoch_broker: refactor ZeroMQ stream for preview: start/end messages always sent

• jfjoch_broker: add crystal lattice plots

• jfjoch_broker: remove empty bins from the plots

• jfjoch_broker: Fix bugs in ModuleSummation and MXAnalyzer for CPU “long” summation

• jfjoch_broker: Fix bug when mean background estimation / indexing rate where affected by previous experiment

• jfjoch_writer: fix missing “-w” parameter

• jfjoch_writer: temporary files have “.tmp” suffix

• jfjoch_writer: refactor logic for watermarks

• jfjoch_writer: report on internal FIFO utilization

• jfjoch_writer: clean-up naming for azimuthal integration and background estimate

• jfjoch_writer: write final background estimate and indexing rate in the master file

• tools/: remove unnecessary tools, make naming consistent

• CBOR: Add indexing rate and background estimate to end message

• CBOR: Clean-up documentation

1.0.0-rc.25

• Updates to documentation

• License set to GPLv3 / OHL-S

• Fix bug in DiffractionExperiment::GetDefaultPlotBinning() - resulting in division by 0 if image time longer than 500ms

• Add information on JUNGFRAU conversion and geometry transformation to CBOR and HDF5

1.0.0-rc.24

New FPGA functionality:

• EIGER supports 8, 16 and 32-bit data input (for 8-bit mode at half performance; for 32-bit “real” depth is 23-bit + 1-bit signed)

• Output possible to 8, 16 and 32-bit data

• Threshold is applied before summation

• Pixel mask can be applied on FPGA

• Mark pixels with ADC content = 0 as bad pixels

• FPGA stores semantic version information (access via /sys/class/misc/jfjoch…/version)

New software functionality:

• Long summation (above 256 frames) done on CPU

• Mechanism to save arbitrary data to HDF5 file

• ZeroMQ preview has option to send start message

• Rework pixel mask + add statistics displayed in web interface

Bug fixes:

• Web frontend: Update preview image automatically during data acquisition

• jfjoch_broker: Error handling if CUDA driver is not installed

• jfjoch_broker: Correctly update progress during pedestal

• jfjoch_broker: Provide proper error when uploaded file is not a proper TIFF

• jfjoch_action_test: enable HLS simulation

Documentation improvement and placement in a dedicated directory