CLAWRIM Wiki

Computing Lab for Advanced Water Resources Informatics and Modeling

User Tools

Site Tools


shell_scripting

Shell scripting

  1. Create C:\opt\bin
  2. Copy busybox.exe to C:\opt\bin
  3. Start busybox.exe
C:\opt\bin\busybox.exe sh -l

Merging CSV files in the current directory

# set a variable to know if a file is the first one
# because we don't want to skip the head line from the first file
first=1
# for each of *.csv file names, assign it to i
for i in *.csv; do
  # if you repeat this script, we don't want to merge
  # the previously merged file again
  if [ "$i" = "merged.csv" ]; then
    # skip this file
    continue
  fi
  # if this file is first
  if [ $first -eq 1 ]; then
    # print everything
    cat "$i"
    # subsequent files won't be first anymore
    first=0
  else
    # for non-first files, skip the first line
    tail -n +2 "$i"
  fi
done > merged.csv # forward any outputs from the above for loop
                  # to merged.csv

Finding the order of a column in a CSV file

head -1 merged.csv |
sed 's/"//g; s/,/ /g' |
awk '{
  for(i=1; i<=NF; i++)
    if($i == "TOBS")
      print i
}'

Counting non-null records in a column in a CSV file

tail -n +2 merged.csv |
sed 's/"//g; s/,/ /g' |
awk '{
  if($57 != "")
    count++
}
END{
  print count
}'
shell_scripting.txt · Last modified: 2023-09-21 05:15 pm by hcho

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki