When I had been given the better part of 8gb of logs recently and was playing with a trial licence of splunk, I was in between a rock and a hard place. I had a string to search for, but wouldn’t be able to ingest that amount of log content within the tight time-frame.
So, enter grep! If you’re new to linux tools, grep searches the contents of files for text. It can even search binary files if you use the right switch, but i’ll tell you about that in a moment.
So I had a string to search for right? Here’s the syntax:
grep -r “10.1.1.1” Webserver\ Logs/ > IPoutput.txt
So what is all this then? Let’s break it down:
grep -r (the -r switch is recursive, so it will go through all of the upcoming directories)
I’ve put the string that i’m searching for into inverted commas “10.1.1.1”
This bit is tricky. I typed ‘Webser’ without the apostrophes, and then pressed TAB. This is the biggest trick in Linux. TAB performed an autocomplete for me, including the escape character ‘\’. That means because there was a space, the ‘\’ means ignore the space.
‘>’ simply redirects the output from the screen to the txt file IPoutput.txt. I’m a Windows person so I’m used to putting extensions on the ends of my files.
I then went from 7.5gb of logs to about 10mb, which was easily imported into Splunk. Happy Days! Hope you have a great weekend.