Showing posts with label Scripts. Show all posts
Showing posts with label Scripts. Show all posts

Download RDS DB logs through AWS CLI

If you need a quicker way to download RDS DB logs, you can do so through AWS CLI (command line interface).  

Although the AWS console also allows downloading these logs, there are disadvantages to this.  Since the console is a graphical user interface, you must click/select the RDS instance first, click on the Monitoring tab of the instance, scroll down to the list of instance logs, and even sort the list of logs in descending order to get to the most recent one -- if you are in the middle of an incident and need to look at the logs to diagnose the issue.  Then you click on the selected log, click the download button, which will bring up a page where you can right-click the selected log using  "save as".  In short, too many mouse clicks before you can get to the needed logs.

Another disadvantage is that you are only allowed to download the log files as they are.  So if the log file is large, you cross your fingers that the network connection will hold steady and won't disconnect in the middle of the download.  If that happens, you have to resume the download and start again.

For faster and more reliable download of RDS DB log files, you can use AWS CLIs instead.  Here is how it is done in Linux.  The same CLIs are also available for Windows.


First, you run the describe-db-log-files command and combine it with the grep Linux OS command to list the existing log files.  You must supply the value for the db-instance-identifier parameter, in this example it is dba-works-01. 

dbaworks@dbasrv1:~$ aws rds describe-db-log-files --db-instance-identifier dba-works-01 | grep "LogFileName"

This will return the list of RDS log files for the dba-works-01 instance.


            "LogFileName": "error/postgresql.log.2021-05-16-12",

            "LogFileName": "error/postgresql.log.2021-05-17-00",

            "LogFileName": "error/postgresql.log.2021-05-17-12",

            "LogFileName": "error/postgresql.log.2021-05-18-00",

            "LogFileName": "error/postgresql.log.2021-05-18-12",

            "LogFileName": "error/postgresql.log.2021-05-19-00",

            "LogFileName": "error/postgresql.log.2021-05-19-12",


Next, run the download-db-log-file-portion  command to begin downloading the selected log file. Again, you should supply the value for the db-instance-identifier parameter, in this example it is dba-works-01, the value for the log-file-name parameter, in this case it is "error/postgresql.log.2021-05-19-12". The next parameter -- starting-token -- allows you to specify where to start paginating the log file.  In this example the value of 0 is used, which means to download the entire file.  The last parameter is output and we use text to specify that the log file should be formatted as text when it is downloaded.  Lastly, we use the Linux pipe to save the results to a local file name dba-works-01-postgresql.log.2021-05-19-12.log


dbaworks@dbasrv1:~$ aws rds download-db-log-file-portion --db-instance-identifier dba-works-01 --log-file-name "error/postgresql.log.2021-05-19-12" --starting-token 0 --output text > dba-works-01-postgresql.log.2021-05-19-12.log


You can also download only the latest portion of the log file by using a simplified command, skipping the -- starting-token parameter, see below: 


dbaworks@dbasrv1:~$ aws rds download-db-log-file-portion --db-instance-identifier dba-works-01 --log-file-name "error/postgresql.log.2021-05-19-12" --output text > dba-works-01-postgresql.log.2021-05-19-12.log

 

The  download-db-log-file-portion command is a nifty tool.  You can run the command with the help option to get more help or information on the other different ways to use this command for your convenience.


aws rds download-db-log-file-portion --db-instance-identifier help



Want to learn more about AWS Command Line Interface (CLI)?  Update your knowledge with this book below:


Labels: , , , ,

Add a Command Alias to Linux User Profile




Problem:  I'm tired of typing long commands in the Linux command prompt whenever I want something done. Not only that, typing these commands is prone to typo errors.

Solution: Add a command alias to our user profile.

Here's how:

1.  Edit your Linux OS user profile (or .bash_profile)  and add a command alias

dbaworks@dbasrv1:~$ vi .profile

-- add the command alias

# set PATH so it includes user's private bin directories
PATH="$HOME/bin:$HOME/.local/bin:$PATH"

alias check_mongo='ps -ef | grep mongo'


2.  Reload your OS user profile

dbaworks@dbasrv1:~$ . .profile

3.  You can now run the command alias and get the desired result:

dbaworks@dbasrv1:~$ check_mongo

--- output
dbaworks@dbasrv1:~$ check_mongo 
mongod 9655 1 1 Jun29 ? 04:15:13 /usr/bin/mongod -- configsvr --replSet csReplSet1 --port 27050 -f /data/mongod.conf --bind_ip_all 


Learn more about Linux shell scripting using this guide book:

Labels: , ,

Shell Script to Automatically Create an AMI Backup of an EC2 Instance (AWS)




Requirements: 

- Access to a Linux machine where shell script will be created
- Access keys to an Amazon Web Services (AWS) Account where the EC2 Instance is hosted


Script name: ami-bash.sh

Input: Name of the instance

How to run: ami-backup.sh <instance_name>

What it does: 

1.  Asks for INSTANCE_NAME

2.  Sets the LOG directory and LOGFILE to record execution

3.  Runs the AWS CLI describe-instances to extract the INSTANCE_ID

4.  Sets the name of the AMI IMAGE_NAME  

5.  Runs the AWS CLI create-image to create the AMI

6.  Runs a loop to check if the image has been created and available

Finished result: An AMI backup of the instance


Script:
=================================================================================
# name:  ami-backup.sh
# usage: ami-backup.sh <instance_name>
#        ami-backup.sh my-db-instance

INSTANCE_NAME=$1

LOGDIR=/tmp/log
LOGFILE=${LOGDIR}/ami-backup-$INSTANCE_NAME-$(date +"%Y-%m-%d-%H-%M-%S").log

INSTANCE_ID=`aws ec2 -describe-instances --filters \
    "Name=tag:Name,Values=$INSTANCE_NAME" \
    | grep "InstanceId" \
    | grep -Po ': "\K[^"]*'`


IMAGE_NAME=ami-backup-$INSTANCE_NAME-$(date +"%Y-%m-%d-%H-%M")

echo "$(date +"%Y-%m-%d-%H:%M:%S") ami-backup.sh started." >> $LOGFILE
echo "$IMAGE_NAME AMI will be created." >> $LOGFILE

IMAGE_ID=`aws ec2 create-image --instance-id $INSTANCE_ID \
     --name "$IMAGE_NAME" \
     --description "$INSTANCE_NAME AMI - $(date +"%Y-%m-%d-%H-%M")" \
     | grep -Po ': "\K[^"]*'`

while [ true ]
do
   echo "AMI creation in progress..." >> $LOGFILE
   IMAGE_CREATION_STATUS=`aws ec2 describe-images --filters \
      "Name=name,Values=$IMAGE_NAME" | grep "State" | grep -Po ': "\K[^"]*'`
      echo "AMI $IMAGE_NAME created." >> $LOGFILE
      break
   fi
   sleep 300
done

echo "$(date +"%Y-%m-%d-%H:%M:%S") ami-backup.sh completed." >> $LOGFILE


=================================================================================


 

=================================================================================

Amazon Photo by Igor Shalyminov on Unsplash
  
































Labels:
,
,
,
,

















        

      









Subscribe to:
Posts (Atom)