Thursday, 22 August 2013

How to merge multiple data sets and append comments to a new variable

How to merge multiple data sets and append comments to a new variable

I have multiple data sets (hundreds) with time series data like this:
"File name";"18%MC001.TXT";"V 1.24"
"Title comment";"231020124070"
"Trigger Time";"'13-04-05 13:53:51"
"Ch";"A 1- 1";"A 1- 2";"A 1- 3";"A 1- 4";"A 1- 5";"A 1- 6";"A 1- 7";"A 1-
8";"A 1- 9";"A 1-10";"A 1-11";"A 1-12";"A 1-13";"A 1-14";"A 1-15";"A 2-
1";"A 2- 2";"A 2- 4";
"Mode";"Voltage";"Voltage";"Voltage";"Voltage";"Voltage";"Voltage";"Voltage";"Voltage";"Voltage";"Voltage";"Voltage";"Voltage";"Voltage";"Voltage";"Voltage";"Voltage";"Voltage";"Voltage";
"Range";"10V";"10V";"10V";"10V";"10V";"10V";"10V";"10V";"10V";"10V";"10V";"10V";"10V";"10V";"10V";"10V";"10V";"10V";
"Comment";"Prove1";"Prove1";"Prove2";"Prove2";"Prove3";"Prove3";"Prove4";"Prove4";"Prove5";"Prove5";"Prove6";"Prove6";"Prove7";"Prove7";"Prove8";"Prove8";"Prove9";"Prove9";
"Scaling";"Off";"Off";"Off";"Off";"Off";"Off";"Off";"Off";"Off";"Off";"Off";"Off";"Off";"Off";"Off";"Off";"Off";"Off";
"Ratio";" 1.00000E+00";" 1.00000E+00";" 1.00000E+00";" 1.00000E+00";"
1.00000E+00";" 1.00000E+00";" 1.00000E+00";" 1.00000E+00";" 1.00000E+00";"
1.00000E+00";" 1.00000E+00";" 1.00000E+00";" 1.00000E+00";" 1.00000E+00";"
1.00000E+00";" 1.00000E+00";" 1.00000E+00";" 1.00000E+00";
"Offset";" 0.00000E+00";" 0.00000E+00";" 0.00000E+00";" 0.00000E+00";"
0.00000E+00";" 0.00000E+00";" 0.00000E+00";" 0.00000E+00";" 0.00000E+00";"
0.00000E+00";" 0.00000E+00";" 0.00000E+00";" 0.00000E+00";" 0.00000E+00";"
0.00000E+00";" 0.00000E+00";"-3.00000E+00";"-3.00000E+00";
"Time";"1-1[V]";"1-2[V]";"1-3[V]";"1-4[V]";"1-5[V]";"1-6[V]";"1-7[V]";"1-8[V]";"1-9[V]";"1-10[V]";"1-11[V]";"1-12[V]";"1-13[V]";"1-14[V]";"1-15[V]";"2-1[V]";"2-2[V]";"2-4[V]";"Event";
0,000000000E+00; 8,69500E-01; 4,80350E+00; 3,76000E-01; 7,34950E+00;
5,60750E+00; 4,66450E+00; 8,31600E+00; 8,13950E+00; 6,66050E+00;
9,69700E+00; 1,81750E+00; 1,10900E+00; 6,82400E+00; 4,04900E+00;
9,82150E+00; 6,98000E+00; 2,94750E+00; 4,08750E+00;0;
1,000000000E+01; 8,69500E-01; 4,80350E+00; 3,76000E-01; 7,34950E+00;
5,60750E+00; 4,66500E+00; 8,31600E+00; 8,13950E+00; 6,66050E+00;
9,69700E+00; 1,81700E+00; 1,10900E+00; 6,82400E+00; 4,04900E+00;
9,82150E+00; 6,98000E+00; 2,94750E+00; 4,08800E+00;0;
Each data set has a unique datetime value (Trigger Time) that is treated
as a comment. Each data set also has a Time variable indicating the time
that has passed since the datetime in Trigger Time. What I want to do is
calculate the datetime for each observation, so that I can graph the data
as a time series using R Statistics. Is there a way to accomplish this?
Merging the datasets and appending the comments does not necessarily have
to be done in R.
I have successfully imported the data from all files in R Statistics using
the list.files and llply functions, as suggested by Matt Bogard in this
blog post. I think I then need to do something similar to what learnr is
suggesting here, but so far my attempts at extracting Trigger Time and
adding a new variable with Trigger Time for each observation have been
unsuccessful.
Solving the problem with Open Refine causes the program to chrash every
time I try to load all the data sets. R may not be the best tool for
processing text files, but I don't have any experience with Python, Ruby
or similar languages.

No comments:

Post a Comment