Early in the 3.x days, metasploit had support for using databases through plugins. As the project grew, it became clear that tighter database integration was necessary for keeping track of the large amount of information a pentester might encounter during an engagement. To support that, we moved database functionality into the core, to be available whenever a database was connected and later added postgres to the installer so that functionality could be used out of the box. Still, the commands for dealing with the database and information stored there were sort of second-class citizens, all beginning with a "db_" prefix. We recently addressed this issue for the upcoming 4.0 release.
Commands that query the database have lost their "db_" prefix, while those that deal with managing the DB itself have retained it. For example, "db_hosts" is now just "hosts" and "db_status" remains the same. The idea behind this change is that hosts (and other entities) don't really have anything to do with the database other than the fact that they are stored there. Additionally, the deprecated db_import_*, db_create, and db_destroy have been removed.
The remaining commands have been improved by expanding search abilities and standardizing option parsing. So where you previously had to type full IP addresses to list more than one host, now all commands that search the database take hosts in nmap host specification format, and all of them that deal with services can take ports similarly. Furthermore, the options have been standardized a bit so -p always means port, -s always means service name.
Example usage for the services command:
msf > services 192.168.1-10.1,3,5 -p 22-25,80,443,445 192.168.99.0/24 Services ======== host port proto name state info ---- ---- ----- ---- ----- ---- 192.168.99.1 22 tcp ssh open 192.168.99.141 445 tcp smb open Windows XP Service Pack 2 (language: Unknown) (name:XP-SP2) (domain:WORKGROUP) 192.168.100.129 445 tcp smb open Unix Samba 3.4.7 (language: Unknown) (name:FOO) (domain:FOO) msf >
The new changes also make it really easy to find services running on odd ports
msf auxiliary(ssh_version) > services -s ssh Services ======== host port proto name state info ---- ---- ----- ---- ----- ---- 192.168.17.134 21 tcp ssh open SSH-2.0-OpenSSH_4.4 192.168.17.134 22 tcp ssh open SSH-2.0-OpenSSH_4.4 192.168.17.134 23 tcp ssh open SSH-2.0-OpenSSH_4.4 192.168.17.134 80 tcp ssh open SSH-2.0-OpenSSH_4.4 192.168.17.134 443 tcp ssh open SSH-2.0-OpenSSH_4.4 192.168.17.134 1433 tcp ssh open SSH-2.0-OpenSSH_4.4 192.168.17.134 8080 tcp ssh open SSH-2.0-OpenSSH_4.4 192.168.17.134 8443 tcp ssh open SSH-2.0-OpenSSH_4.4 192.168.17.134 9022 tcp ssh open SSH-2.0-OpenSSH_4.4 msf >
An often requested feature is the ability to run a module against hosts in the database that match certain criteria. That is now possible for scanner modules with the hosts and services commands' new -R flag (and --rhosts) which sets RHOSTS to the list of hosts returned. If the result is more than 5 hosts, it makes options pretty hard to read, so Metasploit writes it out to a temporary file like so:
msf auxiliary(ssh_version) > services -s ssh --rhosts Services ======== host port proto name state info ---- ---- ----- ---- ----- ---- 192.168.87.1 22 tcp ssh open SSH-2.0-dropbear_0.52 192.168.87.119 22 tcp ssh open SSH-2.0-OpenSSH_5.8p1 Debian-1ubuntu3 192.168.87.122 22 tcp ssh open SSH-2.0-OpenSSH_5.3p1 Debian-3ubuntu6 192.168.87.126 22 tcp ssh open SSH-2.0-OpenSSH_5.1p1 Debian-6ubuntu2 192.168.87.140 22 tcp ssh open SSH-2.0-OpenSSH_5.5p1 Debian-4ubuntu5 192.168.87.145 22 tcp ssh open SSH-2.0-OpenSSH_5.1p1 Debian-6ubuntu2 192.168.87.158 22 tcp ssh open SSH-2.0-OpenSSH_5.3p1 Debian-3ubuntu6 192.168.88.1 22 tcp ssh open SSH-2.0-dropbear_0.52 192.168.89.1 22 tcp ssh open SSH-2.0-dropbear_0.52 192.168.90.1 22 tcp ssh open SSH-2.0-dropbear_0.52 192.168.90.61 22 tcp ssh open SSH-2.0-OpenSSH_5.3p1 Debian-3ubuntu6 192.168.93.1 22 tcp ssh open SSH-2.0-dropbear_0.52 192.168.96.1 22 tcp ssh open SSH-2.0-OpenSSH_5.3p1 Debian-3ubuntu7 192.168.96.134 22 tcp ssh open SSH-2.0-OpenSSH_4.7p1 Debian-8ubuntu1 192.168.98.131 22 tcp ssh open SSH-2.0-OpenSSH_5.1p1 FreeBSD-20080901 RHOSTS => file:/tmp/msf-db-rhosts-20110722-19191-18zr3bq-0 msf auxiliary(ssh_version) > show options Module options (auxiliary/scanner/ssh/ssh_version): Name Current Setting Required Description ---- --------------- -------- ----------- RHOSTS file:/tmp/msf-db-rhosts-20110722-19191-18zr3bq-0 yes The target address range or CIDR identifier RPORT 22 yes The target port THREADS 254 yes The number of concurrent threads TIMEOUT 30 yes Timeout for the SSH probe
Another way to make dealing with all that data easier is through the use of workspaces. Workspaces have been around for awhile, but they are an underused feature that allows you to seperate hosts, credentials, etc. for each engagement into their own silo. Every piece of data that metasploit records is associated with the current workspace, so it's quite easy to keep related information together and segregate different engagements by switching workspaces.
The command by itself will list available workspaces, the current one marked with an asterisk:
msf > workspace default * engagement_A engagement_B engagement_C the_whole_friggin_internet
You can change the current workspace with workspace <name>. For extra convenience, names are tab-completable, too. You can add new workspaces with -a or delete existing ones with -d. Note that -d assumes you really meant it and will happily delete the whole thing (including hosts, credentials, loot, and all) without prompting.
The journey from a glued-on appendage, to a main feature only used by db_autopwn, to a core feature integrated with the whole framework has been an adventure. I think the result is easier access to information, better seperation of that data, and a smoother, faster pentest.