bookmark_borderParse JSON Responses using Bash Scripts

I was working with the categorify.org site and I wanted to parse through the API’s response. The response was in JSON format.

There are a number of different ways you can do this, but if you want a quick, simple, way that uses existing tools you probably already have installed, this is for you.

To parse through the JSON response I used Python.

Here is an example of what I was doing:

curl -s https://categorify.org/api?website=pornhub.com

If you do this, the response is something like this:

$ curl -s https://categorify.org/api?website=pornhub.com

{“domain”:”pornhub.com”,”ip”:”31.192.120.36″,”country-code”:”NL”,”country”:”Netherlands”,”rating”:{“language”:true,”violence”:false,”nudity”:true,”adult”:true,”value”:”R & NSFW”,”description”:”Adult-only content and not safe for a work environment”},”category”:[“Adult\/Pornography”],”keyword_heatmap”:{“videos”:99,”free”:85,”pornhub”:61,”porn”:55,”models”:54,”premium”:45,”content”:45,”rated”:44,”photos”:44,”exclusive”:43,”party”:43,”gifs”:42,”sister”:39,”discover”:36,”online”:36,”video”:35,”albums”:34,”pornhubcom”:33,”cancel”:33,”subscribed”:30}}

What I specifically wanted was the category value: “category”:[“Adult\/Pornography”]

So I modified my command to pipe my output to python and ran the following:

$ curl -s https://categorify.org/api?website=pornhub.com | python -c ‘import json,sys;obj=json.load(sys.stdin);print obj[“category”]’

The section you’re most interested in is this: print obj[“category”]

You have to choose which object you want to print. In my case, I wanted the category object as it holds the value I am looking for.

You can use this to parse any JSON response, just update the print object with one that corresponds to your results. The beautiful part is that this should be available on most major OS platforms without additional installations.


Side note, here are two variations depending on which version of Python you have on your machine.

Python 3

python3 -c "import sys, json; print(json.load(sys.stdin)['category'])"

Python 2

python2 -c "import sys, json; print json.load(sys.stdin)['category']"

Site note, to the side note, if you simply want a pretty display of the json you can pipe the output into “| python3 -m json.tool“. This will give you something much easier to read:

$ curl -s https://categorify.org/api?website=pornhub.com | python3 -m json.tool                                                         {                                                                                                                                                                     "domain": "pornhub.com",                                                                                                                                          "ip": "31.192.120.36",                                                                                                                                            "country-code": "NL",                                                                                                                                             "country": "Netherlands",                                                                                                                                         "rating": {                                                                                                                                                           "language": true,                                                                                                                                                 "violence": false,                                                                                                                                                "nudity": true,                                                                                                                                                   "adult": true,                                                                                                                                                    "value": "R & NSFW",                                                                                                                                              "description": "Adult-only content and not safe for a work environment"                                                                                       },                                                                                                                                                                "category": [                                                                                                                                                         "Adult/Pornography"                                                                                                                                           ],                                                                                                                                                                "keyword_heatmap": {                                                                                                                                                  "videos": 99,                                                                                                                                                     "free": 85,                                                                                                                                                       "pornhub": 61,                                                                                                                                                    "porn": 55,                                                                                                                                                       "models": 54,                                                                                                                                                     "premium": 45,                                                                                                                                                    "content": 45,                                                                                                                                                    "rated": 44,                                                                                                                                                      "photos": 44,                                                                                                                                                     "exclusive": 43,                                                                                                                                                  "party": 43,                                                                                                                                                      "gifs": 42,                                                                                                                                                       "sister": 39,                                                                                                                                                     "discover": 36,                                                                                                                                                   "online": 36,                                                                                                                                                     "video": 35,                                                                                                                                                      "albums": 34,                                                                                                                                                     "pornhubcom": 33,                                                                                                                                                 "cancel": 33,                                                                                                                                                     "subscribed": 30                                                                                                                                              }                                                                                                                                                             }                           

bookmark_borderUse Bash Script to Monitor The Status of Service

When you manage multiple servers it’s sometimes impossible to stay ahead of the various administrative tasks, which is why automation is so important.

If you’re working on a linux based server and want to monitor the status of a service, here is a quick an easy way to automate that process.

#!/bin/bash
# Script to find if a service is running

for i in ossec-monitord ossec-logcollector ossec-integratord;

 do ps auwx | grep -v grep | grep $i >/dev/null 2>&1 ;

   if [ $? = 0 ];

    then

     echo `date “+%Y-%m-%d %H:%M “`”$i Running…”;

    else

     echo `date “+%Y-%m-%d %H:%M “`”$i not running…”;

fi;

What we did above is create a simple loop looking for three distinct services:

  • ossec-monitord
  • ossec-logcollector
  • ossec-integratord

Those three services are assigned to the i variable, and that variable is then passed into the grep query here:

ps auwx | grep -v grep | grep $i >/dev/null 2>&1 ;

What we’re also doing above is cutting out any grep inquiries, because if you were to run a grep command it will show as a process as shown here in red:

root@server:~/scripts# ps auwx | grep ossec-logcollector

root      1260  0.0  0.0   4876  1784 ?        S    May19   0:03 /var/ossec/bin/ossec-logcollector

root     29353  0.0  0.0  14428  1116 pts/0    S+   20:59   0:00 grep –color=auto ossec-logcollector

By cutting out the grep request you see this response:

root@server:~/scripts# ps auwx | grep -v grep | grep ossec-logcollector

root      1260  0.0  0.0   4876  1784 ?        S    May19   0:03 /var/ossec/bin/ossec-logcollector

This is important because this   if [ $? = 0 ]; is looking for the grep exit value of 0, which states:

EXIT STATUS
    The grep utility exits with one of the following values:

    0     One or more lines were selected.
    1     No lines were selected.
    >1    An error occurred.

With this selection, if you run the grep and there are no service running it would still find the grep service itself. It’d give you a false positive response.

The echo command then prints the status of the service:

echo `date “+%Y-%m-%d %H:%M “`”$i Running…”;

It passes each argument through the loop. If the service is found to be running it prints:

2020-05-19 18:27 ossec-monitord Running…

2020-05-19 18:27 ossec-logcollector Running…

2020-05-19 18:27 ossec-integratord Running…