Changelog

1.0.0-rc.68

This is an UNSTABLE release.

  • jfjoch_broker: Temperature threshold can be changed for JUNGFRAU detector

  • jfjoch_broker: Default detector settings can be configured for each detector separately

  • jfjoch_broker: Refactor spot filtering code, max spot count can be modified for dataset settings

  • jfjoch_broker: Refactor indexing refinement, make it the same for both FFBIDX and FFT indexing

  • jfjoch_broker: Reference unit cell will be taken into account for FFT indexing to filter

  • jfjoch_broker: Review PONI rotation angles and azimuthal angle conventions along with PyFAI

1.0.0-rc.67

This is an UNSTABLE release.

  • jfjoch_broker: Enable SSL

  • jfjoch_broker: Wilson B-factor only provided is fit is relatively OK (R^2 > 0.3); this will be refined much more in the future

1.0.0-rc.66

This is an UNSTABLE release.

  • jfjoch_broker: Indexers operate as thread pool, which is operating

  • jfjoch_viewer: Increase interval between loading images + fix too many verbose messages

1.0.0-rc.65

This is an UNSTABLE release.

  • jfjoch_broker: Print information regarding used image pushers

  • jfjoch_viewer: Allow syncing with Jungfraujoch server

  • OpenAPI: Clarify licensing terms in the file

1.0.0-rc.64

This is an UNSTABLE release.

  • jfjoch_broker: Fix issue in receiver light with very long preparation time for threads

  • jfjoch_broker: Add verbose option

  • jfjoch_broker: Don’t trigger pedestal if critical settings are not changed when loading detector settings

  • jfjoch_broker: Detector left in busy state when detector settings were improper

  • jfjoch_viewer: Modify DBus interface to avoid loading same file and image 0 multiple times

  • jfjoch_lite_perf_test: Add verbose option

1.0.0-rc.63

This is an UNSTABLE release.

  • jfjoch_broker: Save NX/NY for grid scan result

  • jfjoch_broker: Add processing time to CBOR output and plot

  • jfjoch_writer: Add processing time to data file

1.0.0-rc.62

This is an UNSTABLE release.

  • jfjoch_broker: Fix bug where low resolution spots were not counted properly

  • jfjoch_broker: Spot count is provided prior to filtering of spots to max_spot_count

  • jfjoch_broker: Add more spot count information to CBOR

  • jfjoch_viewer: Fix issue with ROI drawing resulting in multiple overlapping rectangles

1.0.0-rc.61

This is an UNSTABLE release.

  • jfjoch_broker: Fix bug where FFT indexing could result in a very short or even zero length vector

  • jfjoch_broker: Ice ring and indexed spot count enabled as plots and saved in grid scan results

  • jfjoch_broker: High resolution limit for low res. spot counting can be adjusted

1.0.0-rc.60

This is an UNSTABLE release.

  • jfjoch_broker: Fix bug when the neural network inference client was busy and this status was never released

  • jfjoch_broker: Revert the indexing threshold with distance from integer for Miller indices

  • jfjoch_broker: Fix bug in scattering vector calculation, resulting in indexing not working outside 1.0 A X-ray wavelength

1.0.0-rc.59

This is an UNSTABLE release.

  • jfjoch_broker: Fix bug when broker was waiting for notification message before sending end message, resulting in deadlock.

  • jfjoch_writer: Verbose option for debugging.

1.0.0-rc.58

This is an UNSTABLE release.

  • jfjoch_viewer: Fix memory leak

  • jfjoch_writer: Add detector_number/serial_number to master file

1.0.0-rc.57

This is an UNSTABLE release.

  • jfjoch_broker: Fix bug when enabling ML resolution estimation was not possible

  • jfjoch_viewer: “Movie” mode

1.0.0-rc.56

This is an UNSTABLE release.

  • jfjoch_broker: Fixing more bugs related to neural network inference for ML estimation

1.0.0-rc.55

This is an UNSTABLE release.

  • jfjoch_broker: Fixing minor bugs related to neural network inference for ML estimation

1.0.0-rc.54

This is an UNSTABLE release.

  • jfjoch_broker: Indexing with AUTO settings (FFBIDX if unit cell provided; FFT if not)

  • jfjoch_broker: Don’t remove shared memory area when deactivating detector

  • jfjoch_writer: Save writer release

  • jfjoch_viewer: Increase time for the messages in the status bar

1.0.0-rc.53

This is an UNSTABLE release.

  • PCIe driver: Imperfect solution for RHEL 9.5+ changes

  • jfjoch_writer: Fix to angle containers for AutoProc compatibility

  • jfjoch_fpga_test: Use consecutive number for devices, not interleaved

1.0.0-rc.52

This is an UNSTABLE release.

  • jfjoch_viewer: Use warmer colors to distinguish from AareGUI

  • jfjoch_viewer: Minor adjustments to DBus setting image number

  • jfjoch_broker: Fix in low resolution spot count plotting

1.0.0-rc.51

This is an UNSTABLE release.

  • jfjoch_broker: Send preview in PNG format

  • jfjoch_broker: Provide count of spots in 50.0 - 5.0 A range

  • jfjoch_broker: Provide ML resolution estimation in scan result

  • jfjoch_broker: Allow removing beam center in web preview

1.0.0-rc.50

This is an UNSTABLE release.

  • The release fixes some of many bugs introduced in recent releases

  • jfjoch_viewer: display predictions for indexed cells

1.0.0-rc.49

This is an UNSTABLE release.

  • jfjoch_broker: Handle sample temperature (K) and ring current (mA) to metadata

  • jfjoch_writer: For angle containers in NXmx add _end dataset, sample temp. and ring current

1.0.0-rc.48

This is an UNSTABLE release.

  • jfjoch_broker: fix the bug when a unit cell was not exported for a scan result.

1.0.0-rc.47

This is an UNSTABLE release.

  • jfjoch_viewer: fix dbus service path

  • jfjoch_writer: fix CBF/TIFF writing

1.0.0-rc.46

This is an UNSTABLE release.

  • jfjoch_viewer: remove dependency on image analysis

1.0.0-rc.45

This is an UNSTABLE release.

  • jfjoch_broker: Detector list returns pixel size (mm)

1.0.0-rc.44

This is an UNSTABLE release.

  • jfjoch_broker: more general definition of scan result export

Braking changes:

  • It removes additions to OpenAPI from 1.0.0-rc.43

  • It makes changes to the “unit_cell” definition in OpenAPI specs. It might be harmless in some languages and may result in errors in other implementations.

1.0.0-rc.43

This is an UNSTABLE release.

  • jfjoch_broker: Export grid scan results into a single data structure

1.0.0-rc.42

This is an UNSTABLE release.

  • jfjoch_broker: Add pixel_sum to CBOR output.

  • jfjoch_broker: Changes to sigma estimation in QuickIntegrate routine

  • jfjoch_writer: Save pixel_sum

1.0.0-rc.41

This is an UNSTABLE release. This release includes multiple new features, it should not be used in production at the moment.

  • jfjoch_broker: Estimate B-factor, mosaicity to evaluate crystal diffraction

  • jfjoch_broker: Export GPU count via OpenAPI

  • jfjoch_broker: Enable 2D azimuthal integration and PONI rotations for detector

  • FPGA: Increase the number of integration bins to 2048

1.0.0-rc.40

This is an UNSTABLE release. This release includes multiple new features, it should not be used in production at the moment.

  • jfjoch_broker: Jungfraujoch supports grid scan metadata, including dedicated plotting schemes and NXmx structures

  • jfjoch_broker: Improve metadata for rotation data collection

  • jfjoch_broker: Better handling of plotting

  • jfjoch_broker: FFT based indexing

  • jfjoch_broker: Integration, first try, results not saved at the moment

  • jfjoch_broker: Internal improvements in image handling

  • jfjoch_writer: Multiple adjustments adapt to changes in this release for new features

  • jfjoch_writer: New state management model to improve clarity of error reporting

  • jfjoch_viewer: Remote control via DBus

  • Frontend: Multiple adjustments for new features

  • Frontend: Grid scan plots

WARNING! OpenAPI contains breaking changes in regard to plotting results, so care has to be taken.

1.0.0-rc.39

  • FPGA: Bugfix for pixel masked for data analysis if summation was on

  • jfjoch_viewer: Fix segmentation fault when cursor was outside of image

1.0.0-rc.38

  • jfjoch_broker: Neural net model is not linked with C++ code due to deployment issues, it is rather distributed as python code, connected via RES

  • jfjoch_broker: Neural net model can use all 4 quadrants of the detector

  • jfjoch_broker: For EIGER image time can be provided through /start

  • jfjoch_viewer: Add image list option

  • jfjoch_viewer: Drawing circular ROIs with shift

  • jfjoch_viewer: Enable image summation

  • jfjoch_viewer: Image reader is significantly reworked, hopefully without affecting the viewer

1.0.0-rc.37

  • jfjoch_broker: Make locking rules more flexible

  • jfjoch_broker: Load mask via SIMPLON interface for DECTRIS detectors

  • jfjoch_viewer: Add status bar

1.0.0-rc.36

This is UNSTABLE release. Wait for new version to use in a production environment.

  • jfjoch_broker: Support for Jungfraujoch Lite is enabled - software-based receiver for DECTRIS detectors (required a lot of refactoring, potentially leading to unstable code)

  • jfjoch_broker: Enable Resonet support (ML-based diffraction resolution estimation)

  • jfjoch_broker: Fix error in compression, where bitshuffle/LZ4 and bitshuffle/Zstd HDF5 headers were wrongly generated for 8-bit and 32-bit data

  • jfjoch_writer: Increase buffering to 1000 images in the receiver

  • jfjoch_writer: Images can be written as CBF or TIFF in addition to HDF5

1.0.0-rc.35

This is UNSTABLE release, not properly tested. Wait for new version for using production.

  • jfjoch_broker: If module is delayed by more than 50 frames versus other modules, it will be ignored and receiver is not waiting.

  • jfjoch_writer: Save EIGER energy threshold

  • jfjoch_writer: Add /entry/sample/goniometer for compatibility with eiger2cbf program

1.0.0-rc.34

This is UNSTABLE release - introducing new features, but not properly tested. Wait for new version for using production.

  • jfjoch_broker: More consistency for file format definition (breaking change in API from 1.0.0-rc.31 for file writer settings)

  • jfjoch_broker: For storage cells mask is logical sum of detector bad pixels for all storage cells

  • jfjoch_broker: Handle situation when detector doesn’t want to gracefully stop (to be tested)

  • jfjoch_broker: Center-of-mass position and mean for ROI is added to available plots

  • jfjoch_viewer: Can extract data analysis results from “legacy” format

  • jfjoch_viewer: Display dataset name

  • FPGA: Pixel mask is used for data analysis part even if it is not applied to pixels

  • FPGA: Add pixel sum to module statistics

  • FPGA: ROI number is reduced to 16, but pixel can belong to every defined ROI

  • FPGA: Spot finder is back to full dynamic range (24-bit)

  • FPGA: More debug features for internal FIFOs

Known issues:

  • ROI count flag was added to firmware. For the time being the flag will be wrongly set to 10 due to mismatch of FPGA build scripts.

  • EIGER data acquisition has an issue that is currently debugged

1.0.0-rc.33

  • jfjoch_broker: Fix issue with EIGER settings being loaded improperly

1.0.0-rc.32

  • jfjoch_broker: Refactor code for azimuthal integration for further improvements

  • jfjoch_broker: Minor fix for EIGER (trim energies are manually set for E9M, to be fixed properly later)

  • jfjoch_writer: Fix too much verbose information

  • FPGA: Minor fixes to spot finder (enable two-pass operation and limit number range to int20)

1.0.0-rc.31

This is UNSTABLE release - introducing many features, but still needs more testing. Expecting soon to put bugfix release.

  • jfjoch_writer: Allow to enable overwriting existing files (not enabled by default)

  • jfjoch_writer: Add new HDF5 master file format, which uses HDF5 virtual data sets and links processing results to data files (not enabled by default)

  • jfjoch_viewer: Image viewer work early test version

  • jfjoch_broker: Fixes to counting packets per dataset/image

  • jfjoch_broker: Image buffer is accessible for outside to check images

  • jfjoch_broker: error/saturated pixels and dedicated ROI “beam” can be tracked online

  • jfjoch_broker: Fix bug in handling pedestal G1/G2 count time for JUNGFRAU

  • jfjoch_broker: Fix bug in applying pixel mask interfering with pedestal calculation

  • jfjoch_broker: Fix bug in EIGER initializing

  • jfjoch_broker: Save maximum pixel value to HDF5 file and export as Web plot

  • PCIe driver: Add PCIe link speed and width

  • FPGA: Improve counting error/saturated/min/max pixels

  • FPGA: Spot finder is gradual column-wise (15 columns up/down) and fixed row-wise (32 pixel boxes); previously it was fixed both column- and row-wise with 32x32 pixel areas

  • FPGA: Require Vivado 2022.2

Warning: There are breaking changes to HDF5 file format, renaming entries regarding image storage cell number and image collection efficiency.

1.0.0-rc.30

  • jfjoch_writer: replace non-blocking with blocking operation on internal queues - less likely to “loose” images within the writer

1.0.0-rc.29

  • jfjoch_broker: refactor logic regarding frame time and count time for more flexibility for EIGER and JUNGFRAU

  • jfjoch_broker: readout time for EIGER is 3 us and JUNGFRAU is 20 us, this can be changed in input file

  • jfjoch_broker: OpenAPI interface includes more ways to provide information on the status (error/warning/info)

  • jfjoch_broker: ROIs handling via OpenAPI and frontend is more user friendly

Warning - two breaking changes to OpenAPI:

  • Handling of ROIs is through /config/roi path only for both circle and box ROIs, path in /roi are no longer accessible

  • broker_status structure introduced in 1.0.0-rc.28 has member message and not error_message to allow handling info/warning messages as well

1.0.0-rc.28

  • jfjoch_broker: save error message for initialization and data collection and provide these with OpenAPI

  • jfjoch_broker: fixed issue when in error state, response to /wait_till_done was not complaint to OpenAPI specs

  • jfjoch_test: remove header that failed when CUDA is absent during compilation

  • frontend: add soft trigger button in data collection tab

  • frontend: show error message when in error state

  • CMake: add option to force compilation without CUDA (-DJFJOCH_USE_CUDA=OFF)

1.0.0-rc.27

  • jfjoch_broker: add option to select electron source in instrument metadata, adapt wavelength calculation

  • jfjoch_broker: update pistache web server version

  • jfjoch_writer: minor changes to republish logic

  • Improvements to documentation

1.0.0-rc.26

  • jfjoch_broker: implement ZeroMQ stream for image metadata information

  • jfjoch_broker: refactor ZeroMQ stream for preview: start/end messages always sent

  • jfjoch_broker: add crystal lattice plots

  • jfjoch_broker: remove empty bins from the plots

  • jfjoch_broker: Fix bugs in ModuleSummation and MXAnalyzer for CPU “long” summation

  • jfjoch_broker: Fix bug when mean background estimation / indexing rate where affected by previous experiment

  • jfjoch_writer: fix missing “-w” parameter

  • jfjoch_writer: temporary files have “.tmp” suffix

  • jfjoch_writer: refactor logic for watermarks

  • jfjoch_writer: report on internal FIFO utilization

  • jfjoch_writer: clean-up naming for azimuthal integration and background estimate

  • jfjoch_writer: write final background estimate and indexing rate in the master file

  • tools/: remove unnecessary tools, make naming consistent

  • CBOR: Add indexing rate and background estimate to end message

  • CBOR: Clean-up documentation

1.0.0-rc.25

  • Updates to documentation

  • License set to GPLv3 / OHL-S

  • Fix bug in DiffractionExperiment::GetDefaultPlotBinning() - resulting in division by 0 if image time longer than 500ms

  • Add information on JUNGFRAU conversion and geometry transformation to CBOR and HDF5

1.0.0-rc.24

New FPGA functionality:

  • EIGER supports 8, 16 and 32-bit data input (for 8-bit mode at half performance; for 32-bit “real” depth is 23-bit + 1-bit signed)

  • Output possible to 8, 16 and 32-bit data

  • Threshold is applied before summation

  • Pixel mask can be applied on FPGA

  • Mark pixels with ADC content = 0 as bad pixels

  • FPGA stores semantic version information (access via /sys/class/misc/jfjoch…/version)

New software functionality:

  • Long summation (above 256 frames) done on CPU

  • Mechanism to save arbitrary data to HDF5 file

  • ZeroMQ preview has option to send start message

  • Rework pixel mask + add statistics displayed in web interface

Bug fixes:

  • Web frontend: Update preview image automatically during data acquisition

  • jfjoch_broker: Error handling if CUDA driver is not installed

  • jfjoch_broker: Correctly update progress during pedestal

  • jfjoch_broker: Provide proper error when uploaded file is not a proper TIFF

  • jfjoch_action_test: enable HLS simulation

Documentation improvement and placement in a dedicated directory