Finding things: wildcards, grep, find

Half of data work is finding the right file or the right line in a log. These are the tools.

Wildcards (globs)

The shell expands patterns into matching filenames before the command runs:

• at* — all names that start with at
• *at — all names that end with at
• *at* — all names that contain at
• ? — exactly one character

ls *.csv          # every CSV in this directory
rm tmp_*          # careful — everything starting with tmp_

(Windows people: where CMD used *.*, Linux just uses *.)

grep — search inside files

grep prints the lines that match an expression. You will use it every single day:

grep error app.log            # lines containing "error"
grep -i error app.log         # case-insensitive
grep -v DEBUG app.log         # invert: lines that do NOT match
grep root /etc/passwd         # the classic example

grep understands regular expressions — the same idea you met in the SQL and PySpark courses:

• . matches exactly one character
• .* matches any number of characters
• .+ matches one or more characters

Standard input, output, and pipes

Unix processes read from an input stream (stdin) and write to an output stream (stdout, plus stderr for errors). The pipe | connects one command’s output to the next command’s input:

cat app.log | grep error | head    # first 10 error lines
history | grep ssh                 # find that ssh command you ran last week

This chaining is the whole Unix philosophy: small tools, combined.

find and locate

When you know a file exists but not where:

find ~/projects -name "*.sql" -print   # search a directory tree, live
locate settings.yml                    # search a prebuilt index — much faster

locate uses an index the system rebuilds periodically, so very new files may not appear in it yet.

Practice in your terminal:

1. Run ls *.md (or *.txt) in some project directory.
2. Pick any text file and grep it: grep -i the file.txt.
3. Chain a pipe: history | grep cd — see your own navigation habits.
4. Find every Markdown file under your home: find ~ -name "*.md" | head.