Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
C
CTADIRAC
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package registry
Container Registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Natthan.Pigoux
CTADIRAC
Commits
324886c4
Commit
324886c4
authored
6 years ago
by
Johan Bregeon
Browse files
Options
Downloads
Patches
Plain Diff
remove run_number
move get_run_number to tool_box
parent
84987085
No related branches found
Branches containing commit
No related tags found
Tags containing commit
No related merge requests found
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
Core/scripts/cta-prod-managedata.py
+9
-34
9 additions, 34 deletions
Core/scripts/cta-prod-managedata.py
with
9 additions
and
34 deletions
Core/scripts/cta-prod-managedata.py
+
9
−
34
View file @
324886c4
...
...
@@ -19,28 +19,10 @@ from DIRAC.Core.Base import Script
Script
.
parseCommandLine
()
# Specific DIRAC imports
from
tool_box
import
run_number_from_filename
from
CTADIRAC.Core.Workflow.Modules.Prod3DataManager
import
Prod3DataManager
def
get_run_number
(
filename
,
package
):
"""
try to get a run number from the file name
"""
if
filename
[
-
9
:]
==
'
.logs.tgz
'
:
run_number
=
int
(
filename
.
split
(
'
/
'
)[
-
1
].
split
(
'
_
'
)[
-
1
].
split
(
'
.
'
)[
0
])
elif
package
in
[
'
chimp
'
,
'
mars
'
,
'
corsika_simhessarray
'
]:
run_number
=
int
(
filename
.
split
(
'
run
'
)[
1
].
split
(
'
___cta
'
)[
0
])
elif
package
==
'
evndisplay
'
:
if
filename
[
-
8
:]
in
[
'
DL1.root
'
,
'
DL2.root
'
]:
run_number
=
int
(
filename
.
split
(
'
run
'
)[
1
].
split
(
'
___cta
'
)[
0
])
elif
filename
[
-
10
:]
in
[
'
DL1.tar.gz
'
,
'
DL2.tar.gz
'
]:
run_number
=
int
(
filename
.
split
(
'
run
'
)[
1
].
split
(
'
___cta
'
)[
0
])
else
:
run_number
=
int
(
filename
.
split
(
'
-
'
)[
0
])
# old default
elif
package
==
'
image_extractor
'
:
run_number
=
int
(
filename
.
split
(
'
srun
'
)[
1
].
split
(
'
-
'
)[
0
])
return
str
(
run_number
)
def
put_and_register
(
args
):
"""
simple wrapper to put and register all production files
...
...
@@ -50,13 +32,12 @@ def put_and_register(args):
metadata
=
args
[
0
]
metadata_fields
=
args
[
1
]
file_metadata
=
args
[
2
]
start_run_number
=
args
[
3
]
base_path
=
args
[
4
]
output_pattern
=
args
[
5
]
package
=
args
[
6
]
program_category
=
args
[
7
]
catalogs
=
args
[
8
]
output_type
=
args
[
9
]
base_path
=
args
[
3
]
output_pattern
=
args
[
4
]
package
=
args
[
5
]
program_category
=
args
[
6
]
catalogs
=
args
[
7
]
output_type
=
args
[
8
]
# Load catalogs
catalogs_json
=
json
.
loads
(
catalogs
)
...
...
@@ -78,15 +59,9 @@ def put_and_register(args):
for
localfile
in
glob
.
glob
(
output_pattern
):
file_name
=
os
.
path
.
basename
(
localfile
)
# Check run number, assign one as file metadata if needed
# or add start_run_number
fmd_dict
=
json
.
loads
(
file_metadata
)
if
not
fmd_dict
.
has_key
(
'
runNumber
'
):
run_number
=
get_run_number
(
file_name
,
package
)
fmd_dict
[
'
runNumber
'
]
=
'
%08d
'
%
run_number
else
:
run_number
=
int
(
fmd_dict
[
'
runNumber
'
])
+
int
(
start_run_number
)
fmd_dict
[
'
runNumber
'
]
=
'
%08d
'
%
run_number
run_number
=
run_number_from_filename
(
file_name
,
package
)
fmd_dict
[
'
runNumber
'
]
=
'
%08d
'
%
int
(
run_number
)
# get the output file path
run_path
=
prod3dm
.
_getRunPath
(
fmd_dict
)
lfn
=
os
.
path
.
join
(
path
,
output_type
,
run_path
,
file_name
)
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment