Skip to content

[DF] Redefine()ing a Defined() column causes segmentation violation  #8857

@ShamrockLee

Description

@ShamrockLee
  • Checked for duplicates

Describe the bug

Redefine() is a new method added on the master branch to allow re-definition to an existing column. However, segmentation violation occurs when trying to redefine a defined column.

Expected behavior

Defined columns are re-definable.

To Reproduce

  1. Generate a file containing a simple tree containing a std::vector<Float_t> branch:

gen_tree.C

#include <Rtypes.h>
#include <TFile.h>
#include <TTree.h>
#include <TBranch.h>

#include <vector>
#include <string>
#include <cstdlib>
#include <iostream>

void gen_tree(const std::string pathFile = "file_float_vector.root") {
	TFile *tfOut = TFile::Open(pathFile.c_str(), "RECREATE");
	tfOut->cd();
	TTree *ttOut = new TTree("Events", "Tree of float vectors");
	std::vector<Float_t> vTest(8);
	vTest.clear();
	ttOut->Branch("TestVar", &vTest);
	constexpr size_t nEntries = 100;
	for (size_t jEntry = 0; jEntry < nEntries; ++jEntry) {
		const Float_t var0 = rand() / static_cast<Float_t>(RAND_MAX);
		const size_t nVar = rand() % 8;
		std::cerr << "jEntry: " << jEntry << "\tnVar: " << nVar << "\tvar0: " << var0 <<  std::endl;
		vTest.resize(nVar, 0.);
		for (size_t iVar = 0; iVar < nVar; ++iVar) {
			vTest[iVar] = var0 + 0.125 * iVar;
			std::cerr << vTest[iVar] << " " << std::flush;
		}
		std::cerr << std::endl;
		ttOut->Fill();
	}
	ttOut->Write();
	tfOut->Close();
}
  1. Read with RDataFrame, define a column, redefine it, and save the two columns to histograms:
    reproduce.C
#include <Rtypes.h>
#include <TFile.h>
#include <TTree.h>
#include <TBranch.h>
#include <TH1.h>
#include <ROOT/RDataFrame.hxx>

#include <vector>
#include <string>

void reproduce(const std::string pathFileIn = "file_float_vector.root", const std::string pathFileOut = "file_out.root") {
	TFile *tfIn = TFile::Open(pathFileIn.c_str());
	TTree *ttIn = tfIn->Get<TTree>("Events");
	ROOT::RDataFrame dfIn(*ttIn);
	std::cerr << "Creating a custom column ..." << std::endl;
	auto dfAug = dfIn.Define("TestVar2", "TestVar + 1.");
	std::cerr << "Redefining ..." << std::endl;
	auto dfOut = dfAug.Redefine("TestVar2", "TestVar2.size() ? ROOT::VecOps::Take(TestVar2, {0}) : ROOT::VecOps::Take(TestVar2, {})");
	std::cerr << "Action: Histo1D" << std::endl;
	// auto histview1 = dfOut.Histo1D({"hTest1", "Histogram showing the column in the input tree", 30000, 0., 3.}, "TestVar");
	auto histview2 = dfOut.Histo1D({"hTest2", "Histogram showing the custom (defined) column", 30000, 0., 3.}, "TestVar2");
	std::cerr << "Saving histogram ..." << std::endl;
	TFile *tfOut = TFile::Open(pathFileOut.c_str(), "RECREATE");
	tfOut->cd();
	// std::cerr << "Writing histview1" << std::endl;
	// histview1->Write();
	std::cerr << "Writing histview2" << std::endl;
	histview2->Write();
	std::cerr << "Finalizing ..." << std::endl;
	tfOut->Close();
	tfIn->Close();
	std::cerr << "Complete!" << std::endl;
}

Here is the output of the second piece of code:

$ root -l -q "reproduce.C++()"

Processing reproduce.C++()...
Info in <TUnixSystem::ACLiC>: creating shared library /run/media/root/data-btrfs/shamrock-shared/targets/Research_LL/ctau-proper/lxplus_HTcondor/preselect/redefine_custom_vector/./reproduce_C.so
Creating a custom column ...
Redefining ...
input_line_34:2:99: warning: braces around scalar initializer [-Wbraced-scalar-init]
auto lambda1 = [](ROOT::VecOps::RVec<double>& var0){return var0.size() ? ROOT::VecOps::Take(var0, {0}) : ROOT::VecOps::Take(var0, {})
                                                                                                  ^~~
Action: Histo1D
Saving histogram ...
Writing histview2

 *** Break *** segmentation violation



===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================
#0  0x00007f5ace552106 in wait4 () from /nix/store/cvr0kjg2q7z2wwhjblx6c73rv422k8cm-glibc-2.33-47/lib/libc.so.6
#1  0x00007f5ace4d1cbf in do_system () from /nix/store/cvr0kjg2q7z2wwhjblx6c73rv422k8cm-glibc-2.33-47/lib/libc.so.6
#2  0x00007f5acec98597 in TUnixSystem::StackTrace() () from /home/shamrock/Research_LL/ctau-proper/lxplus_HTcondor/preselect/result/lib/libCore.so
#3  0x00007f5acec95955 in TUnixSystem::DispatchSignals(ESignals) () from /home/shamrock/Research_LL/ctau-proper/lxplus_HTcondor/preselect/result/lib/libCore.so
#4  <signal handler called>
#5  0x00007f5ac53c13e8 in ?? ()
#6  0x0000000000000000 in ?? ()
===========================================================


The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum https://root.cern/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at https://root.cern/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#5  0x00007f5ac53c13e8 in ?? ()
#6  0x0000000000000000 in ?? ()
===========================================================


Root > 
 *** Break *** segmentation violation



===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================
#0  0x00007f5ace552106 in wait4 () from /nix/store/cvr0kjg2q7z2wwhjblx6c73rv422k8cm-glibc-2.33-47/lib/libc.so.6
#1  0x00007f5ace4d1cbf in do_system () from /nix/store/cvr0kjg2q7z2wwhjblx6c73rv422k8cm-glibc-2.33-47/lib/libc.so.6
#2  0x00007f5acec98597 in TUnixSystem::StackTrace() () from /home/shamrock/Research_LL/ctau-proper/lxplus_HTcondor/preselect/result/lib/libCore.so
#3  0x00007f5acec95955 in TUnixSystem::DispatchSignals(ESignals) () from /home/shamrock/Research_LL/ctau-proper/lxplus_HTcondor/preselect/result/lib/libCore.so
#4  <signal handler called>
#5  0x00007f5ace877360 in __dynamic_cast () from /nix/store/lg104nh0szci8slz5z6494m457jm5y3p-gcc-10.3.0-lib/lib/libstdc++.so.6
#6  0x00007f5ab9f1fc0d in TTree::~TTree() () from /home/shamrock/Research_LL/ctau-proper/lxplus_HTcondor/preselect/result/lib/libTree.so
#7  0x00007f5ab9f20349 in TTree::~TTree() () from /home/shamrock/Research_LL/ctau-proper/lxplus_HTcondor/preselect/result/lib/libTree.so
#8  0x00007f5acebeb4f8 in THashList::Delete(char const*) () from /home/shamrock/Research_LL/ctau-proper/lxplus_HTcondor/preselect/result/lib/libCore.so
#9  0x00007f5aceb37e7a in TROOT::EndOfProcessCleanups() () from /home/shamrock/Research_LL/ctau-proper/lxplus_HTcondor/preselect/result/lib/libCore.so
#10 0x00007f5acec9161f in TUnixSystem::Exit(int, bool) () from /home/shamrock/Research_LL/ctau-proper/lxplus_HTcondor/preselect/result/lib/libCore.so
#11 0x00007f5aceb44017 in TApplication::Terminate(int) () from /home/shamrock/Research_LL/ctau-proper/lxplus_HTcondor/preselect/result/lib/libCore.so
#12 0x00007f5acee3c936 in TRint::Run(bool) () from /home/shamrock/Research_LL/ctau-proper/lxplus_HTcondor/preselect/result/lib/libRint.so
#13 0x000000000040110c in main ()
===========================================================


The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum https://root.cern/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at https://root.cern/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#5  0x00007f5ace877360 in __dynamic_cast () from /nix/store/lg104nh0szci8slz5z6494m457jm5y3p-gcc-10.3.0-lib/lib/libstdc++.so.6
#6  0x00007f5ab9f1fc0d in TTree::~TTree() () from /home/shamrock/Research_LL/ctau-proper/lxplus_HTcondor/preselect/result/lib/libTree.so
#7  0x00007f5ab9f20349 in TTree::~TTree() () from /home/shamrock/Research_LL/ctau-proper/lxplus_HTcondor/preselect/result/lib/libTree.so
#8  0x00007f5acebeb4f8 in THashList::Delete(char const*) () from /home/shamrock/Research_LL/ctau-proper/lxplus_HTcondor/preselect/result/lib/libCore.so
#9  0x00007f5aceb37e7a in TROOT::EndOfProcessCleanups() () from /home/shamrock/Research_LL/ctau-proper/lxplus_HTcondor/preselect/result/lib/libCore.so
#10 0x00007f5acec9161f in TUnixSystem::Exit(int, bool) () from /home/shamrock/Research_LL/ctau-proper/lxplus_HTcondor/preselect/result/lib/libCore.so
#11 0x00007f5aceb44017 in TApplication::Terminate(int) () from /home/shamrock/Research_LL/ctau-proper/lxplus_HTcondor/preselect/result/lib/libCore.so
#12 0x00007f5acee3c936 in TRint::Run(bool) () from /home/shamrock/Research_LL/ctau-proper/lxplus_HTcondor/preselect/result/lib/libRint.so
#13 0x000000000040110c in main ()
===========================================================


Error in <TList::Delete>: A list is accessing an object (0x7f5acedf5830) already deleted (list name = ClosedFiles)

  1. Substituting the line
auto dfOut = dfAug.Redefine(...);

with

auto dfOut = dfAug;

and it will complete the execution without errors.

Setup

  1. ROOT version: master (commit eb7c9c4), "6.25/01"
  2. Operating system: NixOS
    • system: "x86_64-linux"
    • host os: Linux 5.10.48, NixOS, 21.05.20210717.b2f87e0 (Okapi)
    • multi-user?: yes
    • sandbox: yes
    • version: nix-env (Nix) 2.4pre20210601_5985b8b
    • nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos
  3. How you obtained ROOT: Build myself by upgrading the Nixpkgs root package

Additional context

Nix expressions used to build ROOT:
flake.nix

{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-21.05";
  inputs.nixpkgs-root.url = "github:ShamrockLee/nixpkgs/root-6-25";
  inputs.flake-utils.url = "github:numtide/flake-utils";
  inputs.root-source.url = "github:root-project/root/master";
  inputs.root-source.flake = false;
  outputs = inputs@{nixpkgs, nixpkgs-root, flake-utils, root-source, ...}: flake-utils.lib.eachDefaultSystem (system: let
    pkgs = nixpkgs.legacyPackages.${system};
    pkgs-root = import nixpkgs-root {
      inherit system;
      overlays = [
        (final: prev: {
          root = prev.root.overrideAttrs (oldAttrs: {
            src = root-source;
          });
        })
      ];
    };
  in{
    legacyPackages = pkgs;
    legacyPackages-root = pkgs-root;
    packages = {
      inherit (pkgs-root) root gcc gnumake cmake;
      inherit (pkgs) gawk;
    };
    defaultPackage = pkgs-root.root;
    devShell = pkgs.mkShell {
      buildInputs = (with pkgs-root; [
        root
      ]);
      nativeBuildInputs = (with pkgs-root; [
        gcc
        gnumake
        cmake
      ]);
    };
  });
}

Generated lock file
flake.lock

{
  "nodes": {
    "flake-utils": {
      "locked": {
        "lastModified": 1623875721,
        "narHash": "sha256-A8BU7bjS5GirpAUv4QA+QnJ4CceLHkcXdRp4xITDB0s=",
        "owner": "numtide",
        "repo": "flake-utils",
        "rev": "f7e004a55b120c02ecb6219596820fcd32ca8772",
        "type": "github"
      },
      "original": {
        "owner": "numtide",
        "repo": "flake-utils",
        "type": "github"
      }
    },
    "nixpkgs": {
      "locked": {
        "lastModified": 1628696776,
        "narHash": "sha256-K8+/IQNZDZltIu/KSf+Tz01OwNzTGgTch0ktMOWcxLg=",
        "owner": "NixOS",
        "repo": "nixpkgs",
        "rev": "927ce1afc1db40869a463a37ea2738c27d425f80",
        "type": "github"
      },
      "original": {
        "owner": "NixOS",
        "ref": "nixos-21.05",
        "repo": "nixpkgs",
        "type": "github"
      }
    },
    "nixpkgs-root": {
      "locked": {
        "lastModified": 1628672385,
        "narHash": "sha256-Xb/yIAo1OK8OEwrtd5SP3bjAD06PLXsrWG3WTLKqme0=",
        "owner": "ShamrockLee",
        "repo": "nixpkgs",
        "rev": "60e709069fa2fbcfca943d8f6df8dc6d33c67025",
        "type": "github"
      },
      "original": {
        "owner": "ShamrockLee",
        "ref": "root-6-25",
        "repo": "nixpkgs",
        "type": "github"
      }
    },
    "root": {
      "inputs": {
        "flake-utils": "flake-utils",
        "nixpkgs": "nixpkgs",
        "nixpkgs-root": "nixpkgs-root",
        "root-source": "root-source"
      }
    },
    "root-source": {
      "flake": false,
      "locked": {
        "lastModified": 1629293709,
        "narHash": "sha256-+blxCSBSr2mt7baeYR3iSLr7dk1pEhhfNsv7I3CDzhs=",
        "owner": "root-project",
        "repo": "root",
        "rev": "eb7c9c44465ad3f8db90d83598e3116a45ca630e",
        "type": "github"
      },
      "original": {
        "owner": "root-project",
        "ref": "master",
        "repo": "root",
        "type": "github"
      }
    }
  },
  "root": "root",
  "version": 7
}

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions