Skip to content

Added ROOT schema evolution rule for obscure versions of blip::Blip - #178

Open
PetrilloAtWork wants to merge 1 commit into
SBNSoftware:developfrom
PetrilloAtWork:feature/gp_blipEvolution
Open

PetrilloAtWork wants to merge 1 commit into
SBNSoftware:developfrom
PetrilloAtWork:feature/gp_blipEvolution

Conversation

@PetrilloAtWork

Copy link
Copy Markdown
Member

Some SBND production data appears to have been produced with a version of blip::Blip and subclasses that are not available in GIT develop branch and whose versions and checksums are not listed in their classes_def.xml.
Attempting to read a ROOT tree containing branches with those objects immediately fails.

This change introduces a ROOT input rule to read such old data and convert it into the current object format. This allows reading those data files with the code in this branch, at the cost of losing blip::HitClust::IntersectLocations information (see below).
Changes in the format mainly include conversion of a few data from TVector3 to geo::Point_t, in the classes:

  • blip::Blip:
    • TVector3 Position → geo::Point_t Position
    • blip::HitClust clusters[3] → std::array<blip::HitClust, kNplanes> clusters
  • blip::HitClust (blip::Blip::clusters):
    • std::map<int,TVector3> IntersectLocations → std::map<int,geo::Point_t> IntersectLocations
  • blip::TrueBlip: TVector3 Position → geo::Point_t Position

My changes make the blip::HitClust::IntersectLocations empty and do not convert the existing data, due to limitations of at least one among ROOT 6.28/12 and the committer. A warning is printed to std::cerr once per job when that happens.
The blip::Blip::clusters C-array itself does not need a special rule since ROOT already knows how to deal with arrays to std::array conversion.

Suggested reviewers:

  • @Jjm321814 (author of the original code)
  • @VCLanNguyen (involved in the production of the data files)
  • a SBND production manager (since this is related to production data)

Technical notes

How I did that

The code of the pre-publishing classes (which went into the production) are not in develop branch and I could not find any branch featuring them.
A schema evolution rule allows to single out an "older" object and to update it to the current version. Normally we associate a version number to a class checksum in classes_def.xml.
A I/O rule can identify which object it applies to with either the version number or the checksum. Unfortunately, classes_def.xml is not available and we can't read neither the version not the checksum from it.
Both can, however, be read from a data file containing a tree with such an object.
This information is stored in the class streamer information, which is stored "hidden" in the ROOT file (i.e. is not included in the TTree object but still travels with it).
The incantation to discover the information from a ROOT file _file0 for an object called blib::Blip is:

dynamic_cast<TStreamerInfo const*>(_file0->GetStreamerInfoList()->FindObject("blip::Blip"))->GetCheckSum()

and changing the string to the appropriate class name reveals the other checksums.
The class version is not exposed, but can be read calling Dump() of the TStreamerInfo object that the line above loaded. It turns out that classes were not assigned any version number, so they all have the default 1. At least, it does not conflict with the current develop versions (10).
The rules use the checksum to identify their object, since version 1 is quite non-descriptive, being the default value.

The checksums were extracted from the file
root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/sbnd/persistent/users/lnguyen/Run2_NarrowBunch_BNBLight_Reco2_v10_14_02_02/data_filtered_decoded_reco1-fb4d5378-4822-3334-eca8-a88fcff8c908_Reco2-20260421T041203.root, which appears to have been produced with a sbndcode version derived from v10_14_02_02 (mostly out of the directory name).
This file does not seem to have metadata stored in the SAM database.

What's wrong with blip::HitClust::IntersectLocations

The I/O rule for blip::HitClust::IntersectLocations includes a one-time warning printout, debugging code activated with gDebug ≥ 2 (for example with Root.Debug: 2 in ${HOME}/.rootrc) and a piece of code permanently disabled (if (false)) showing how the conversion of the map should have worked.

Unfortunately, with ROOT 6.28/12 and GCC 12.1.0 that code shows a map (onfile.IntersectLocations) that is not correctly loaded, with invalid internal pointers and node counts. The debug message will enable people to check whether the values are reasonable when executed with a newer ROOT or GCC (ROOT is now shipping version 6.40), and potentially reenable the conversion. Until then, this rule just produces an empty map.

The source data format is not in GIT, but relevant data was produced with it.
This chacge allows reading that data with a newer release.

The map `blip::HitClust::IntersectLocations` is made empty and not converted,
due to limitations of at least one among ROOT 6.28/12 and the committer.
@Jjm321814

Copy link
Copy Markdown
Contributor

Ran the branch over a _v10_14_02_02 reco2 output and a gen2 production file (reco1 + reco2 + cafmaker).
Both made CAF successfully.
The gen2 file reco2 stage output still has intersection locations added in normally.

I think the loss of old intersection information is fine. It is an internal variable used for blip reco only and probably shouldn't ever be opened by an analyzer. And if they really need the information they can rerun reco2 themselves.

source="TVector3 Position;"
targetClass="blip::TrueBlip"
target="Position"
checksum="[ 2186401263 ]"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does this checksum value come from?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to explain the process in the "How I did it" section of the description. In short: from a sample data file.
How did it end there? From some unsanctioned working area used for some kind of a production.

@Jjm321814

Copy link
Copy Markdown
Contributor

One additional point, will all future versions of the blip/trueblip/hitclust class need the conversions copied in the same way?
For example when we add new energy variables to the class as in #174?

We should get those two PR to work well together before merging them in.

@PetrilloAtWork

Copy link
Copy Markdown
Member Author

Short answer: no extra step is needed for any future variation of this class.

These rules affect only the input with the specified checksum, and only on reading. When using other input, the newer code (that is, code that has a version of the data classes newer than the one in the input) will write the new classes applying the standard I/O rules that we never needed to worry about — according to those rules, you will get a value for that "energy" that is from the default constructor of the object.

However, make sure to merge this PR before the other one, so that the latter determines the newest class version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working maintenance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants