Differences

This shows you the differences between two versions of the page.

--- shell_scripting [2023-09-21 04:41 pm] – hcho
+++ shell_scripting [2023-09-21 05:15 pm] (current) – hcho
@@ Line 12: / Line 12: @@
 <code bash>
+# set a variable to know if a file is the first one
+# because we don't want to skip the head line from the first file
 first=1
+# for each of *.csv file names, assign it to i
 for i in *.csv; do
+  # if you repeat this script, we don't want to merge
+  # the previously merged file again
   if [ "$i" = "merged.csv" ]; then
+    # skip this file
     continue
   fi
+  # if this file is first
   if [ $first -eq 1 ]; then
+    # print everything
     cat "$i"
+    # subsequent files won't be first anymore
     first=0
   else
+    # for non-first files, skip the first line
     tail -n +2 "$i"
   fi
-done > merged.csv
+done > merged.csv # forward any outputs from the above for loop
+                  # to merged.csv
 </code>
+===== Finding the order of a column in a CSV file =====
+<code bash>
+head -1 merged.csv |
+sed 's/"//g; s/,/ /g' |
+awk '{
+  for(i=1; i<=NF; i++)
+    if($i == "TOBS")
+      print i
+}'
+</code>
+===== Counting non-null records in a column in a CSV file =====
+<code bash>
+tail -n +2 merged.csv |
+sed 's/"//g; s/,/ /g' |
+awk '{
+  if($57 != "")
+    count++
+}
+END{
+  print count
+}'
+</code>