From 51a3d3de2299190cb18b5367bd4dcc4e9a2d1bf6 Mon Sep 17 00:00:00 2001 From: fyodor Date: Mon, 13 Apr 2026 20:09:30 +0000 Subject: [PATCH] Remove todo directory as it hasn't really been used in more than 10 years and we keep these in issues DB now --- todo/david.txt | 42 - todo/djalal.txt | 146 -- todo/dmiller.txt | 12 - todo/done.txt | 3711 ------------------------------------- todo/gorjan.txt | 66 - todo/henri.txt | 41 - todo/nmap.txt | 638 ------- todo/nping.txt | 799 -------- todo/patrick.txt | 77 - todo/paulino.calderon.txt | 4 - todo/sctp.txt | 49 - todo/shinnok.txt | 150 -- 12 files changed, 5735 deletions(-) delete mode 100644 todo/david.txt delete mode 100644 todo/djalal.txt delete mode 100644 todo/dmiller.txt delete mode 100644 todo/done.txt delete mode 100644 todo/gorjan.txt delete mode 100644 todo/henri.txt delete mode 100644 todo/nmap.txt delete mode 100644 todo/nping.txt delete mode 100644 todo/patrick.txt delete mode 100644 todo/paulino.calderon.txt delete mode 100644 todo/sctp.txt delete mode 100644 todo/shinnok.txt diff --git a/todo/david.txt b/todo/david.txt deleted file mode 100644 index a9ce685c1..000000000 --- a/todo/david.txt +++ /dev/null @@ -1,42 +0,0 @@ -* Make improvements to the irc-unrealircd-backdoor script. -* Brandon says: "Sometime -sV goes just a little too fast and gets a connect - error. It should back off and try again a few times before giving up trying - to fingerprint the service." It looks like - Got nsock CONNECT response with status ERROR - aborting this service - Add a delay of 500 ms? -Summer of coder: -* Add a library function to test the randomness of a string. Use it to make - version scripts for services that send random or encrypted data, for example - cccam on port 12000 which sends 16 bytes. - -Zenmap: -* Do a memory audit of loading a large scan file. -* Figure out what licensing notices are required in the Mac package for GTK+, - Glib, Python, and anything else we use. -Summer of Coder: -* Merge a scan aggregation into one XML file. -* Synthesize text Nmap output from an XML file. - -Ncat: -* Make Ncat send one line at a time when --delay is in effect. This is - cumbersome to do until Nsock supports buffered reading. -* Make the HTTP proxy support the chunked transfer encoding, then change it to - be HTTP/1.1 and support pipelining. -* See if we can make Ncat drop privileges on startup. - -Nsock: -* Add a buffer to each iod, so that you can ask for a certain number of bytes - or lines and get exactly that many, no more. Venkat wrote a proposal at - http://seclists.org/nmap-dev/2009/q3/0600.html. - -Web site: -* Look for a good online respository viewer. - -Done: -* Handle multiple targets with the same address. -* Check necessity of mswin32 pcap includes. -* Try removing the call to PacketSetReadTimeout in readip_pcap, so that Windows - uses the short 2 ms timeout like some other platforms without selectable pcap - fds do. Measure difference in time and CPU time. -* Do JavaScript magic to expand/contract NSEDoc sidebar. -* Check out compression options for the NSIS installer. diff --git a/todo/djalal.txt b/todo/djalal.txt deleted file mode 100644 index b674d16ee..000000000 --- a/todo/djalal.txt +++ /dev/null @@ -1,146 +0,0 @@ -== - -GSoC 2011 TASKS: - -o Work on my GSoC vulnerability and exploitation script ideas: - https://secwiki.org/w/Nmap/Script_Ideas#Djalal_Harouni - -o Review all the "Improve NSE HTTP architecture" proposal suggetions - and comments, and try to include them and update the proposal. - http://seclists.org/nmap-dev/2011/q2/967 - -o Start a thread on Nmap-dev about users favorite Nmap and NSE commands, - and create a special page for it in the secwiki.org site. - This will also let us to create more scan profiles for Zenmap. - -== - -1) Nmap Scripting Engine Infrastructure: - -o [High priority] - Take a look at Dan's NSE XML output patch and try to commit it. - http://seclists.org/nmap-dev/2011/q2/1230 - -o NSE Version Numbering. - http://seclists.org/nmap-dev/2010/q4/693 - -[Other tasks] -o Propose a better duplicate scanned IPs filtering engine. - - -2) NSE Scripts: - -[Priorities tasks] -o NFS/RPC features: -- add NFS READLINK support to let nfs-ls show symbolic files. - -o Review NSE scripts and libs, and fixing bugs: - - Document all the new NFS procedures. - -[Other tasks] -o NFS/RPC features: -- Add more authentication support: Unix authentication. -- NFSv4 support. -- Add recursion support to nfs-ls.nse - - -== - -MAYBE: - -o Create a new rule "versionrule" which will be used by version - category scripts. - http://seclists.org/nmap-dev/2010/q3/551 - -o NSE debugger. - -o Add more NSE control for long running scripts: one option will be a -boolean expression filter (like: tcpdump) which will change NSE scripts -arguments or behaviour according to previous results, this will be -really useful for big networks. Another option will be a generic NSE -(Lua) script with an easy and readable code that includes expressions or -filters selection to let us change NSE arguments according to previous -results. -Note: this option will be useful on big networks. however for the moment -this is a simple idea and it needs further discussion on the nmap-dev. - -o Privileges dropping for NSE scripts [nmap TODO list]. - -o NSE security review [nmap TODO list]. - - -o Fixing bugs. -- NSE not honoring the source port flag when doing version scan. - http://seclists.org/nmap-dev/2010/q2/576 - - David said that it will not be easy to support setting the source port - http://seclists.org/nmap-dev/2010/q3/331 - - -== - -DONE: - -1) Nmap Scripting Engine Infrastructure: - -o Submitted the "Improve NSE HTTP architecture" proposal - http://seclists.org/nmap-dev/2011/q2/967 - -o Make NSE scripts able to retrieve the interface network - information. - -o LuaFileSystem directory iterator [1] port. -[1] http://keplerproject.github.com/luafilesystem/ - -o New class of scripts which use two new script rules: - - Script Pre-scanning and Script Post-scanning rules: "prerule" and - "postrule". Documented these new phases. - - Update scripts to use these new rules: - dns-zone-transfer now uses "prerule" and "portrule". - -o Update other parts of Nmap book to show the new Script scan phases. - -o Fixing bugs: - - NSE not honoring the Exclude directive bug fixed and committed - as r18467. - -o Let NSE "prerule", "portrule" and "hostrule" scripts to add new -discoverd targets to Nmap. - -o Update scripting.xml to show the new script scan phases. - - -2) NSE Scripts: - -o smtp-vuln-cve2011-1764 script to check Exim DKIM Format String - vulnerability (CVE-2011-1764). - -o Updated and Improved ftp-vsftpd-backdoor to detect the vsFTPd backdoor - (CVE-2011-2523). - -o ftp-vuln-cve2010-4221.nse script to check the ProFTPD Telnet IAC stack - overflow (CVE-2010-4221). - -o smtp-vuln-cve2010-4344 script to check and exploit Exim SMTP Server: - heap overflow (CVE-2010-4344) and privileges escalation (CVE-2010-4345) - -o SMTP library. - -o Rewritten SMTP scripts to use the smtp library: - - smtp-commands - - smtp-open-relay - - smtp-enum-users - -o smtp-vuln-cve2011-1720 script to check for CVE-2011-1720 - -o broadcast-avahi-dos script to check for CVE-2011-1002 - -o NFS/RPC features: - - New script: nfs-ls which combines nfs-dirlist and nfs-acls and try to - emulates some features of the old "ls" unix tool. The script support - NFSv2 and NFSv3. - - Readapted the RPC and NFS library code with a new re-design with new - high level functions. - - Added NFS procedures support: - NFSv2: LOOKUP - NFSv3: FSSTAT, FSINFO, READDIRPLUS, PATHCONF, ACCESS, LOOKUP diff --git a/todo/dmiller.txt b/todo/dmiller.txt deleted file mode 100644 index 0216a3c3e..000000000 --- a/todo/dmiller.txt +++ /dev/null @@ -1,12 +0,0 @@ -* Make Zenmap unit tests work. Guessing lots don't, since r32569 fixed real code - that matched some unit tests, too. - -* Make sure Ndiff, Zenmap are 2to3 compatible with python -3 - -* Script to check for updated versions of included libs. Have shell for libpcap, - but should convert to python. - -* NSE stuff - * broadcast-srvloc-info - test - * broadcast-rpcbind - write, test - * Consolidate utility functions diff --git a/todo/done.txt b/todo/done.txt deleted file mode 100644 index ccb0a1ca2..000000000 --- a/todo/done.txt +++ /dev/null @@ -1,3711 +0,0 @@ -DONE: - -o Change Ncat so that it does SSL certificate trust checking by - default (even without --ssl-verify) and provides a warning and the key - fingerprint if there is no valid trusted chain or the cert is - expired, etc. The warning should happen (to STDERR) even if -v is - not specified. We should add a new option to force Ncat to quit if - cert not valid, and --ssl-verify should become an undocumented alias - for that. [GH#30] - -o Augment the configure script to list unmet dependencies. Currently, configure - works just fine without a C++ compiler installed, but make generates an - error. The configure script should be able to detect this. Also, a list of - features that are/are-not available would be nice at the end of the script, - so folks can see that they've e.g. missed the OpenSSL dependency. - -o Add parallel IPv6 reverse DNS support (right now we use the system - functions). - -o [Ncat] This may sound ridiculous, but I'm starting to think that - Ncat should offer a very simple built-in http server (e.g. for simply - sharing files, etc.) And maybe a simple client too. (Done via --lua-exec and - the httpd.lua script shipped with Ncat) - -o INFRASTRUCTURE: Add IPv6 support to secwiki - - We probably just have to designate a new IPv6 address for it and - add it to Apache config. - -o [INFRASTRUCTURE] Improve our main web server http configuration to - better handle high load situations and DoS attacks. As part of - this, we may have to raise the max client limits. But then there is - a risk of running out of RAM, which can be even worse. So we need - to figure out a good balance. - -o Migrate web.insecure.org to a RHEL-6 derived distro (probably CENTOS - 6, since Linode doesn't currently offer ScientificLinux images). - o Actually, if we can wait until "second half of 2013", we might be - able to jump straight to RHEL 7. And RHEL 5 support looks like it - will go on for many more years for critical/security patches. - o Maybe start with svn server, since we've had reports of our - current one giving people unexpected password prompts. There is a - thread about that at http://seclists.org/nmap-dev/2012/q2/17 - o UPDATE on this - adding read-only rights (rather than no rights) - to the root of the svn repo seems to have solved this problem. - -o Make Windows 8.1 VM with VS 2013 and do more testing of Nmap compilation/running - -o Make and test build on a newer OS X than 10.6 (10.10 was recently released) - -o Adopt an issue tracking system for Nmap and related tools. We - should probably look at our needs and options and then decide on and - either install it on our own infrastructure or use it hosted elsewhere. - - David notes that Trac seems to work well for Tor -- see - https://trac.torproject.org/projects/tor - - One thing which can be nice is being able to interact with the - system through email. Like for bugs people file on the Nmap package - in Debian, I can just reply to the mail and it gets added in the tracker. - - This is now live at http://issues.nmap.org/ - -o Update OpenSSL library to 1.0.1j - -o Our "make uninstall" should uninstall ndiff if it was installed too. -  We should probably do it in pretty much the same way we handle -  Zenmap (configure.ac, Makefile.in, and ndiff/setup.pl) - -o Web: We should probably distribute RapidSSL intermediate certificate - on SecWiki so it is trusted even if browsers don't have that cert - cached. Here's a page nothing the issue: - https://www.ssllabs.com/ssltest/analyze.html?d=secwiki.org - - We probably need to add an entry in apache conf after - SSLCertificateFile which looks something like: - SSLCertificateChainFile /etc/apache2/rapidssl.pem - -o The XML version of Nmap lists and describes the six port states - recognized by Nmap near the top of the "Port Scanning Basics" - section.  That can be seen in the HTML rendering at - https://nmap.org/book/man-port-scanning-basics.html.  But in the man - page (nroff) rendering, the list is missing and it just gives the - title: "The six port states recognized by Nmap".  UPDATE: Now the - descriptions for each state appear in the man page, but the headings - ("open", etc.) are missing. We should figure out - why, and fix it. - - The bug in the stylesheets means that (From Daniel): "if you have an - element and it's followed by anything other than whitespace+CDATA - (like " foo") then the remaining cdata or element until - the next new element will be nroff-commented so this - blah is ok, but this blah, is not ok because of the commaand this blah nmap -A is bad no matter how much whitespace intervenes" - - -o Fix a segmentation fault in Ncat when scanned with the SSL NSE - scripts. I was able to reproduce this on 2013-09-27 with latest SVN - by running: - Ncat: ncat -v -k --ssl -l localhost - Nmap: ./nmap --script-trace --script '+ssl*' localhost -p 31337 - This was initially reported by Timo Juhani Lindfors on the Debian - bug tracker: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=724580 - Henri notes: "I traced the latter back to openssl and opened a - ticket there, which never got any reply... https://rt.openssl.org/Ticket/Display.html?id=2885&user=guest&pass=guest" - -o Investigate how we're ending up with OS fingerprints in nmap-os-db - with attribute names like W0 and W8 when according to the docs they - are only supposed to be W1 - W6 (and plain W). - https://nmap.org/book/osdetect-methods.html#osdetect-w. See also - http://seclists.org/nmap-dev/2013/q4/68. Need to determine how - these are getting into the file (from Nmap itself or our - integration/merge tools) and fix that then remove them from the - file. - -o Integrate latest IPv4 OS detection submissions and corrections - -o We should improve the Windows build process for Ndiff, since it - works differently now that it is modularized. To build the Nmap - 6.45 release, we (as a temporary hack, not in SVN): - - Added 'ndiff' to zenmap/setup.py 'packages' list in - COMMON_SETUP_ARGS - - Created a zenmap/ndiff subdir (empty) and copy ndiff/ndiff.py into zenmap/ before build. - We should find a more elegant solution and check it into SVN. The - fundamental issue is that the ndiff.exe we generate needs to be - able to access the new ndiff.py module. - Also, we need to make sure the -win32.zip Nmap distribution works - properly. - -o [Zenmap] Combine parallel timed-out hops into one node in the - topology view. http://seclists.org/nmap-dev/2012/q1/82 has a patch, - however it doesn't handle the case of two or more consecutive - timeouts. - -o If Nmap uses a "tcpwrapped" port to do fingerprinting on, OS detection - might give false matches/results. Since it doesn't really matter which - open port gets chosen, we should move onto another open port if we - notice "tcpwrapped". - -o Implement an --exclude-ports option. See - http://seclists.org/nmap-dev/2012/q1/275 - -o In an ideal world, Zenmap would not run out of memory and crash. - And we already have an entry for improving Zenmap's memory - consumption. But in the meantime, we should catch the error and - present a more useful error message/explanation so the user - understands the problem. This should reduce the number of - out-of-memory "crash reports" we get too. See - http://seclists.org/nmap-dev/2014/q2/298 - -o Provide an option to send a comment in scan packet data for target - network. Examples: --data-string "Scan conducted by Marc Reis from - SecOps, extension 2147" or --data-string "pH33r my l3eT - s|<iLLz! I'll 0wN UR b0x!" - -o We should probably update our included libpcap. We currently - include version 1.2.1 (we upgraded to that in April 2012) while the - latest version on tcpdump.org is 1.5.3. We make minor changes to - libpcap that we ship, and instructions for upgrading are in - libpcap/NMAP_MODIFICATIONS. - -o Investigate report of Nmap ARP discovery using the wrong target MAC - address field in ARP requests (it is correct in the ethernet frame - itself). See this thread: http://seclists.org/nmap-dev/2011/q3/547 - -o Add randomizer to configure script so that a random ASCII art from - docs/leet-nmap-ascii-art*.txt is printed. I think I'll start naming - them leet-nmap-ascii-art-submittername.txt. - -o Add IPv6 subnet/pattern support like we offer for IPv4. - o OK, we now have the subnet/pattern support, but not the two-stage - model discussed below. So we added a separate task for that. - o Obviously we can't go scanning a /48 in IPv6, but small subnets do - make sense in some cases. For example, the VPS hosting company - Linode assigns only one IPv6 address per user (unless they pay) - and you can find many Linode machines by scanning certain /112's. - And patterns might be useful because people assigned /64's might - still put their machines at ::1, ::2, etc. - o David says: "We need to design a new way to iterate over host - specifications (i.e., different than nexthost). Because the new - host discovery code is sometimes going to want whole netblocks - and sometimes individual hosts. So I'm thinking of a two-stage - model, where the iterator will received (parsed) specifications - like AAAA::1/48, and then it can decide whether to further - iterate that into individual addresses, or pass the block off - to some specialized discovery routine." - - -o Consider implementing RPC scan with ultra_scan or something else. - Right now it is the only program using pos_scan. On the other hand, - I'm not sure TCP RPC scanning is appropriate for ultra_scan. - -o When Ncat is compiled without OpenSSL, we should still accept the - --ssl argument and just give an error message noting that SSL was not - compiled in. This reduces confusion for users - (e.g. http://seclists.org/nmap-dev/2013/q3/579) - -o We should update our OpenSSL Windows binaries from version 1.0.1c to - something newer, like 1.01f - -o Web: figure out why autogeneration of nmap.org/nsedoc/ doesn't seem - to be working. I think we had a cron job which was supposed to be - doing it. - - hb system was still running crontab files from old web vm in its - rc.local. Fixed. - -o Add a W3C XML Schema Definition (XSD) for Nmap XML output. Keeping the DTD - around is also helpful, but XSD is widely supported and could help improve - support for Nmap XML in other tools. - o We're going to discuss this on mailing list before deciding - whether to 1) switch from DTD to XSD, 2) stick with just a DTD, or - 3) try to support both. - -o Update copyright year to 2013 in the Nmap copyright header files - -o Update CHANGELOG for new release - -o New Nmap Release - -o Nping in ICMP mode (default) must not be checking the icmp IDs or - returned packets or something, because if I have two separate 'nping - scanme.nmap.org' running at the same time, each nping sees the replies - from the other nping (as well as its own) and it screws up the timing - stats too. - -o Process Nmap OS service detection submissions - - New fingerprints + corrections - - Last done November 2012: http://seclists.org/nmap-dev/2012/q4/222 - -o Process Nmap IPv6 OS detection submissions - - New fingerprints + corrections - -o Process Nmap IPv4 OS detection submissions - - New fingerprints + corrections - - Last done in November 2012: http://seclists.org/nmap-dev/2012/q4/221 - -o Make Ncat reset the signal handler for SIGPIPE to SIG_DFL before - execing a program with --exec and friends. A "broken pipe" error in - a subprocess should kill the subprocess. Lack of default SIGPIPE - handling is what prevents a trivial Lua chargen script--it loops - forever after the socket disconnects because none of its writes - fail. Cf. http://www.chiark.greenend.org.uk/ucgi/~cjwatson/blosxom/2009-07-02-python-sigpipe.html. - -o [Nping] In '-q' mode, Nping should keep the line giving the min/max/avg rtt - times. That way people can avoid seeing each individual packet but - still see the stats which are similar to what normal ping gives - them. - -o [Nping] Remove the lines starting with 'Tx time' and 'Rx time' by - default (and of course quieter modes), but leave them for cases at - least one level of -v. - -o Nping/Nmap should probably show ICMP ping sequence values by default - in packet trace mode. This would be nice for Nping since that is - the default ping it sends and is the main way to distinguish the - packets since the IPIDs are the same. - -o Complete migration away from Syn colocated machine - - [Done - actually was already on web] Move submission CGIs to web - - Make sure notification still works - - [Done] Mailman - - [Done] Install mailman software on web, including CGIs - - Migrate mailing lists to web - -o Remove the -q/FAKE_ARGV stuff from Nmap, since I don't think people - use that any more. - -o We should document Ron's sample script - (https://nmap.org/svn/docs/sample-script.nse) in docs/scripting.xml - so that new script writers know about it. - - Decided to remove it instead. Justification: "It is a great idea, - but nobody seems to use it (for example, there were no replies to - usage inquiry here: http://seclists.org/nmap-dev/2012/q4/379). I - think there are two main uses for this script, both of which are - being served by other resources. 1) as a template for new - scripts. Users instead seem to pick a script that is most similar - to the one they want to write and start with that. 2) As a way to - learn more about the format of an NSE script. Users instead seem - to use our documentation - (https://nmap.org/book/nse-script-format.html). So I'm deleting it - for now. But if folks miss it, they're welcome and encouraged to - say so on dev@nmap.org and we could consider putting it back - and/or improving it" - -o Upgrade Mac Mini to Mac OS X 10.8 (Mountain Lion) and test building - as well as testing usage of our normal builds (which we currently - build on 10.6). - -o Make a branch from the 6.20BETA1 release (r30266) for new stable - release, apply any important bugfix patches from the meantime and then - release it after Thanksgiving as new Stable release. - -o [NSE] We may want to consider a better exception handling method -- - one which doesn't require wrapping every I/O line in its own try - function call. David says "Lua has an internal "exception handling" - mechanism based on a function called pcall, which is implemented - with setjmp/longjmp. You can wrap a function call in it and the - function will return there whenever there's an unhandled error. - Something based on that would be better [than the current system], I - think." - - This one is obsolete as the Lua 5.2 now lets you do a Lua yield - across C function calls. - -o Add IPv6 support to Nping, including raw packet mode (hopefully - sharing as much code with Nmap as possible, though Nping's packet code - is a bit different), and also including echo mode server and client - support. - -o Make sure we update everywhere relevant (e.g. refguide, etc.) to - note the addition in Nmap of the Liblinear library for large linear - classification (http://www.csie.ntu.edu.tw/~cjlin/liblinear/). It - uses a three-clause BSD license: - http://www.csie.ntu.edu.tw/~cjlin/liblinear/COPYRIGHT - - David has added it to 3rd-party-licenses.txt - - Fyodor moved it into the refguide - -o Consider including OpenSSL in our Nmap tarball - - Need to check the size, etc. - - OK, we're counting this as done because we took all the Win - binaries out of the tarball and put them in an nmap-mswin32-aux svn - directory which users check out to compile Nmap on Windows, and - OpenSSL is included in this. - -o Update the Nmap CHANGELOG for latest improvements - -o Do an Nmap dev release. Last release was Nmap 6.01 June 22. - o Update Nmap version number and auto-generated files for release. - -o Process latest Nmap OS submissions and corrections (IPv4 and IPv6). - Last done (for IPv4 anyway) in February 2012. - -o Review and consider integrating Tomas Hozza's UNIX-domain socket - support patch for nsock/ncat: http://seclists.org/nmap-dev/2012/q4/24. - -o Improve CPE coverage in OS detection DB from 84% to 90% (see CPE - entry a ways down for more on this). - -o Process latest service detection submissions. They were last done - in February 2012. - -o Integrate Henri's new kqueue/poll nsock-engines support. - -o If it is trivial to add, it would be nice if the "New VA Module - Alert Service" also gave the Author field for NSE scripts so everyone - knows which hero(es) wrote it. - -o Clean up the Nmap repo to remove some bloat we've allowed to creep - in. Should do a more thorough search, but for now here are two - obvious candidates: - - Create publicly readable /nmap-mswin32-aux in svn - - Files not needed for compiling Nmap itself (e.g. only needed for - creating or including in Nmap packages), particularly including the - vcredist files, should be moved to new /nmap-mswin32-aux - - The /nmap-mswin32-aux files won't be included in Nmap tarballs - either - - Add the gtk, glib, etc. Windows dependencies to /nmap-mswin32-aux - so users don't have to all install those in order to compile Zenmap - and make Nmap packages. - - move the nmap-private-dev/mswin32 stuff into /nmap-mswin32-aux - - Update nmap-install.xml for new changes. Such as noting need to - checkout this new directory for building packages, removing the - need to install your own gtk, glib, etc. - - [done] Remove the 5MB of XSL in nping/docs/xsl - -o Update our mswin32/OpenSSL to newest version (previous update was - September 2010 to 1.0.0a). - -o Nmap should have a better way to handle XML script output. - o done: https://nmap.org/book/nse-api.html#nse-structured-output - o We currently just stick the current script output text into an XML tag. - o Daniel Miller is working on an implementation: - https://secwiki.org/w/Nmap/Structured_Script_Output - -o Update more web content in real time (or near real-time, or at least - on an automated basis rather than requiring manual checkin and - update). In particular: - o NSEDoc generation - o [done] SVN dir (https://nmap.org/svn/) should be removed and a redirect - added to https svn server. - o Maybe Nmap book building - o Maybe the generated files in nmap.org/data/ - -o Update web.insecure.org so that rather than requiring us to build - nsedoc on other machines, check it into svn, and then update svn on - web, it is done by a script on web which could be run through cron - (and potentially from a simple svn commit hook) to build them on the - web server directly. - - There are other similar things we might want to automate later, - such as book rebuilding when the XML files are changed. - -o Investigate/fix potential routing-related issue. See emails from - Djalal and others: http://seclists.org/nmap-dev/2012/q3/116, - http://seclists.org/nmap-dev/2012/q3/4, - http://seclists.org/nmap-dev/2012/q2/449 - -o Even without the --osscan-guess flag, Nmap should show the closest - matches (if they pass our threshold) in the XML output. We omit - them from the normal output in large part to encourage people to - submit fingerprints, but that argument doesn't apply so well to XML - output users. Normal output users who really want to see the Nmap - guesses could still use --osscan-guess as before. - -o Change the interface of nmap.ip_send to take an explicit - destination address. It currently extracts the destination from - the packet buffer, which does not have enough information to - reconstruct link-local addresses. See r26621 for a similar change - that was made to Nmap internals. - -o [Zenmap] Install higher-resolution icons (at least 64x64 and maybe - up to 512x512). Here is a screenshot of the current 48x48 icon on - GNOME 3: http://seclists.org/nmap-dev/2012/q2/395. - o Sean did Windows and Linux icons, and David did the Mac - one. - - -o [NPING] At least on my (Fyodor) system, I get errors like "READ-PCAP - killed: Resource temporarily unavailable" with some commands. - Example: - # nping --tcp -p80 -c1 scanme.nmap.org - - Starting Nping 0.5.61TEST4 ( https://nmap.org/nping ) at 2012-02-16 17:52 PST - SENT (0.3307s) TCP 192.168.0.5:42005 > 74.207.244.221:80 S ttl=64 id=23109 iplen=40 seq=1015357225 win=1480 - RCVD (0.3524s) TCP 74.207.244.221:80 > 192.168.0.5:42005 SA ttl=51 id=0 iplen=44 seq=3197025741 win=14600 - nping_event_handler(): READ-PCAP killed: Resource temporarily unavailable - nping_event_handler(): TIMER killed: Resource temporarily unavailable - [...] - -o [NPING] Nping should probably give you an error or warning when you - do: "nping -p80 google.com" since it is ignoring the port specifier. - The user probably wants to add --tcp. - -o Investigate why http pipelining so often doesn't work in NSE - scripts, and often NSE ends up reverting to one request at a time. - Scripts may not be using it correctly, and also we wish it were more - transparent and there wasn't this big API divide between pipeline - and non-pipeline. We just want it send requests as fast as it can, - and get a callback when there's a response. Maybe the http library - buffers them, or pipelines them, or blocks the http.get call until - there's more room. It just seems to always degenerate to 1 request - at a time. For example: - sudo nmap --script=http-enum bamsoftware.com -p80 -d2 - quickly (within a few seconds) gives: - NSE: http-enum: Searching for entries under path '' (change with 'http-enum.basepath' argument) - NSE: Total number of pipelined requests: 2081 - NSE: Number of requests allowed by pipeline: 100 - NSE: Received only 41 of 100 expected responses. - Decreasing max pipelined requests to 41. - NSE: Received only 1 of 41 expected responses. - Decreasing max pipelined requests to 1. - 100 may a wildly high number of requests to attempt to pipeline. - And then something else probably goes wrong after it decides 41 is okay. - - Related: Does caching work with pipeleined requests? We should - make sure it does. - [ OK, the main part of this todo item is done. Though there is a - patch pending from Piotr which changes how pipelining works that - is worth considering. We did fix the underlying pipelining bug, but - (just as with most browsers), it isn't enabled by default. Also, it - doesn't support caching. See - http://seclists.org/nmap-dev/2012/q3/616. ] - -o Make Nmap from a clean start (e.g. after make clean or whatever, so - it compiles everything) and research all the compile warnings to see - which ones can be fixed/removed. Of course caution is needed to - make sure we don't cause problems. For example, an unused variable - on one platform might not be unused on another, so we can't just - remove it. May have to surround it by ifdefs though. - -o Solve "spurious closed port detection" issue discovered by David: - http://seclists.org/nmap-dev/2012/q1/62 . So we need to figure out - what is going on here and then how to fix it. Note that this - doesn't seem to happen when you do ICMP host discovery first (-PE), - so it probably relates to the ACK packet that Nmap sends to port 80 - on the target by default. - -o Add real headers for more protocol types in -6 -sO scan. Dario - Ciccarone provided some packet captures for - 0x00: hop-by-hop - 0x2b: routing - 0x2c: fragment - 0x3c: destination - (http://seclists.org/nmap-dev/2011/q2/1003). We also have examples - of crafting some of these in FPEngine.cc. [Sean and David] - - -o Investigate increasing FD_SETSIZE on Windows to allow us to - multiplex more sockets. See Henri's email: - http://seclists.org/nmap-dev/2012/q1/267 - [James Rogers did some investigative work on this in July 2012, but - we weren't able to find a great solution. Maybe we should - investigate this more in the future, and also investigate other - Windows socket APIs such as completion ports. ] - -o Finish sv-tidy - a program to canonicalize and tidy nmap-service-probes. - o Check for the same reference (like $1) being used in unrelated fields - (where related fields are the pairs (p, cpe:), (v, cpe:), (i, cpe:), - (o, cpe:)). - For example if we have v/$1/ h/$1/ it is a bug. - o Check a list of common product names that should only appear in p//, - not in i//. We still have entries that are like this: - p/Foobar 2000 ADSL router/ i/micro_httpd web server/ - that should rather be written this way: - p/micro_httpd/ i/Foobar 2000 ADSL router/ - o [Done] Check for e.g. i/French/ without :fr in cpe:/a, and vice versa. - [Sean and David?] - -o Remove Nmap's --log-errors feature and make its behavior the - default. A few notes: - - Nmap should just ignore --log-errors if it sees it - - Remember to remove it from the documentation - -o We should probably sort script output (for port output and host - output) by script name or something so that it comes in a - deterministic order. If the same three scripts produce output in - two different scans, they should be listed in the same order. Right - now the order can vary, at least for host output. - [Sean] - -o Add a function such as --disable-arp-ping which prevents hosts from - being automatically detected as 'up' just because they responded to - ARP. Instead, Nmap will actually send the requested host discovery - probes (ICMP ping packets, SYN packets, etc.) and only mark the host - as up if it responds on an IP level. This is how machines are - already treated if they're not on the local network (e.g. if ARP - discovery is unavailable). This technique is a bit slower and more - likely to miss hosts (e.g. if they're heavily firewalled) than ARP - discovery, but the option is needed to handle local networks which use - proxy ARP, which would otherwise cause all IPs to appear to be up. - -o We should add fields to the service submitter [James is working on this] - (http://insecure.org/cgi-bin/submit.cgi?new-service) for the - application name and version. - o We also need to ensure all fields of /cgi-bin/submit.cgi have - proper escapting to prevent possible reflected XSS attacks - reported by Maxim Rupp (@mmrupp). The risk is low, if any, since - we don't give authentication cookies for bad guys to steal, but is - still better to properly escape. - o If we get a chance, would be interesting to run our XSS-testing - NSE scripts against this and see if they locate the problems. - o Also, need to change the font family in there from "Lucida Grand" - to "Lucida Grande"? Just a typo. And fix "WIkipedai". We should - just spell-check all the output - -o Make Nmap 6.01 release containing (among possibly other little -fixes) - - Python upgrade - - [done] Zenmap 10.7 hang fix (done in trunk) - - [done] Zenmap crash when filtering hosts (done in trunk) - - [done] get_srcaddr fix (done in trunk) - -o Upgrade Python on build machines to try and resolve Python 2.7 - security warning (it doesn't affect us, but can worry users). See - this thread: http://seclists.org/nmap-dev/2012/q2/621 - -o Fix get_srcaddr error happening on Windows XP - -o [Web] Add a page with the Nmap related videos we do have already - - We have a page on Secwiki now: https://secwiki.org/w/Nmap/Presentations - -o Zenmap hang on OS X 10.7 - -o For many years, the Nmap man page and online documentation has had - an "Inappropriate Usage" section which notes that "Nmap should never - be installed with special privileges (e.g. suid root) for security - reasons". And of course Nmap's official installer would never - install Nmap that way. While one would thinks that would be enough, - we might want to go even further and have Nmap detect when it is run - suid and print a security warning. - -o Prepare release notes, web page, etc. - -o Do private beta release - -o Make the release - -o In Nmap XML output, osclass (OS Classification) tags should be - children of osmatch (the human readable OS name line) rather than - having Nmap deduplicate all the osclasses and put them in as - siblings. But this change might break some systems which utilize - Nmap XML output, so, along with this change, we need to introduce an - option such as --deprecated-osclass-xml to return the old behavior. - That option only needs to be documented in the CHANGELOG entry - referring to this change, and it should note that we're likely to - remove this option in a year or two. - -o Right now, when an IPv4 or IPv6 address seems bogus (such as 1.2.3 - or 2001::0 in IPv4 mode), we give a fatal error and abort the scan. - But since that might just be one bad target in a long list of hosts to - be scanned, it is probably better to just print a warning and - continue. Some sort of warning or host element should be included in - the XML to explain what happened too. This should also happen if - we're unable to resolve a DNS name. - -o In sv-tidy, check that used references start at 1 and are - contiguous. If $1 and $3 are used but not $2, it's probably a bug. - Maybe you can even find out how many there should be by inspecting - the regular expression. - -o Raw scans from Mac OS X seems not to retrieve the MAC address or do - ARP ping, except when scanning the router on an interface. For - example, scanning 192.168.0.1-5 sends ARP pings to 192.168.0.1, but - the normal four-probe combination to the other addresses. The "MAC - address:" line appears in the output for .1 but not for the others. - -o To avoid Nmap memory usage bloat, find a way for NSE scripts to - store information about a host which expires after Nmap is done - scanning that host (e.g. when the hostgroup containing that host is - finished). Right now scripts store such information in the registry - and it persists forever. For example, a web spidering - script/library could store information about the web structure and - even page contents so that other scripts can use that information - without spidering the target again, but ensuring that the memory - will be freed after the hostgroup finishes so there is room to store - the web information for the next group of systems. One idea would - be to make a host.registry member which contains a registry specific - to a specific target. Scripts could store temporary information - there, but still use the global registry for information which must - persist (e.g. to be used by postrules, etc.) - -o Add CPE support to IPv6 OS detection - -o Use BPF libpcap logic on Solaris 11, otherwise packet capture doesn't - work at all. http://seclists.org/nmap-dev/2012/q1/613 - -o [NSE] host.os should not just be a list of strings which can contain - human-readible strings and/or CPE info. It should probably be list - of host.os tables which can contain: - host.os[].name <-- human readible name - host.os[].class[].vendor - host.os[].class[].osfamily - host.os[].class[].osgen - host.os[].class[].devicetype - host.os[].class[].cpe[] <-- array of cpe:/ strings - So host.os[1].class[1].cpe[1] is the first CPE entry for the first - classification of the first OS match for the target system. - The host.os entry docs/scripting.xml would have to be updated too. - -o We should probably go through the nmap-os-db (and IPv6 version) - entries and, where the fingerprint line specifies a service pack - number (or even two of them), ensure that we have sp-qualified CPE - entries like "cpe:/o:microsoft:windows_xp::sp2". Right now we - sometimes include the qualification, and sometimes not. - o This is best done with cpeify-os.py, if possible. - -o Zenmap no longer ads the installed module directory to its module - search path because some distributors first install in a world - writeable directory (like /tmp) and then put those files into their - packages which they distribute to users. But this change can lead - to Zenmap not working for users who install in nonsystem areas like - their home directory (e.g. --prefix /home/fyodor) unless they have - their PYTHONPATH set to find them. We should implement a solution, - such as making sure Zenmap catches the missing modules error and - suggest that the user set their PYTHONPATH or something. - -o Scans from Mac OS X tend to use raw IP packets rather than ethernet - frames even on the local network because Dnet does not seem to be - retrieving the routing table properly -- so the LAN doesn't even - show up in --iflist. Patrik can reproduce this on all 3 of his - MACs (OS X versions 10.7.3). Comparing the code in DNet route-bsd.c - to Apple's own routing table code discovered by Patrik suggests that - the Dnet code may be incorrect. - -o ssl-google-cert-catalog should not require that the user specify - ssl-cert in order to run. Instead, they should probably both call a - library which obtains the certificate (and caches it so that it - doesn't happen twice if both scripts are run). In general, we want - to avoid having any scripts tell the user "this script only works if - you specify this other script too". If we really find we need that - functionality, we should add a "strong dependencies" feature so that - scripts can tell Nmap what other scripts they require. - [Patrik did this by adding an ssl cert library] - -o Our targets-ipv6-multicast-slaac.nse should probably send the router - advertisements with low priority to reduce the chances of any - negative impacts on clients, if we're not doing that already. See - http://lists.si6networks.com/pipermail/ipv6hackers/2012-March/000503.html. - - Actually, I think we already do this. Marking as done. - -o Deal with the issue of timeouts happening too soon due to global - congestion control in some cases. For example, if Nmap sends host - discovery probes to two hosts, and one comes back extremely quickly, - it can cause the global congestion control to use a very low timeout - and cause the 2nd host (which doesn't have any host-based congestion - control values yet) to timeout arguably too quickly. We should look - at potential algorithm changes to improve this. - David: I think I was wrong about the cause of this. Even when - replies come back very quickly, the timeout is by default limited - to 100000 microseconds, much higher than the straightforward - calculation would give. What I think is really happening is that - select is not working reliably on this platform (Solaris 10 x86). - In the loop in read_arp_reply_pcap, pcap_select returns 1, then a - pcap_next is done. Then pcap_select returns 0, but if I insert - another pcap_next after that, the pcap_next finds another packet - without blocking (the first time, anyway; after that it blocks). - -o Create CHANGELOG - -o Make stable release candidate branch - -o Make at least one more test release from the candidate branch - -o Write and send GSoC 2011 results email - -o Document the nsearg format changes made by Paulino (how you can - preface an argument with a script to make it more specific, or make it - general to apply to multiple scripts) - o Rough drafts: - o nmap-exp/calderon/refguide.xml - o nmap-exp/calderon/scripting.xml - o Relates to: - o We should probably modify stdnse.get_script_args so that it first - checks [scriptname].[argname] and then (if that fails) looks for - [argname] by itself. This way people who are only running one - script or who want to use the same value for multiple scripts that - take the same argument can just give [argname]. But those who want - an argument to only apply to a specific script can give - [scriptname].[argname]. - -o Make the nmap.header.tmpl wording a little more generic so it more - clearly applies to Ncat, Zenmap, Nping, etc. Then use - templatereplace.pl to apply those changes to the code. [Fyodor] - -o Change Nmap copyright dates (in the file headers, etc.) from 2011 to - 2012. - -o Get RPM staticly linking to libsvn (rather than dynamic linking) so - that it isn't a requirement for installing the RPM. - - We decided to just make nmap-update its own separate RPM so that - it can dynamically link to libsvn without forcing that dependency on - the whole nmap RPM package. - - since the libsvn-devel package apparently only installs dynamic - libs, we'll probably have to install it ourselves on the CentOS - build machines. - -o Fix "BOGUS! Can't parse supposed IP packet" in packet trace of IPv6 - packets. - -o Integrate latest IPv6 OS detection fingerprint submissions - - In addition to the submission CGI submissions, some were emailed to Fyodor and David on Oct 21 - -o Integrate new service fingerprint submissions (we have more than - 2,531 submissions in two files since 11/30/10) - -o Integrate new OS detection submissions (1,893 since 6/22/11) - -o Add options in configure script for users to specify where to find - subversion lib/include dirs (like we do with our other library - dependencies). See this mail: - http://seclists.org/nmap-dev/2012/q1/37 - -- David added --with-apr and --with-subversion - -o We need to fix the svn server so that Nmap committers can make - branches from /nmap to /nmap-exp. We may need to add some sort of - OPTIONS permission to the root directory or something, because - they're getting errors like: - $ svn cp https://svn.nmap.org/nmap https://svn.nmap.org/nmap-exp/branchname - svn: Server sent unexpected return value (403 Forbidden) in response - to OPTIONS request for 'https://svn.nmap.org' - - Patrick also reported some other funny business related to svn - mv'ing directories in email to Fyodor and David. - -o Give CPE visibility to NSE. - - done by Henri - -o Document the new IPv6 OS detection novelty system in os-detection.xml - -o Do more thinking/researching/investigating the way our machine - learning IPv6 OS detection system decides whether a match is perfect - and/or how close the match is. Maybe our current system works well - enough, we'll need to watch how it performs as we increase the DB - size and collect/integrate more signatures. The goal is to: - o Producing fewer way-off matches since it would have a way (like our - current system) to decide how close the match really is - o Doing a better job about printing fingerprints for matches with - aren't close enough - -o Improve the "run Zenmap as root" menu item to work on distributions - without su-to-root. We might even want to improve Zenmap so that it - itself does not have to run as root, and just executes Nmap that - way. Rather than not showing Zenmap as root on the Menu of - non-working systems, it might be better to have it but let it give - an error message (and then, perhaps, run as nonroot) so that users - of those distributions are more likely to contribute a fix. We also - might want to look at how the distributions themselves package Zenmap. - -o Consider changing Nsock so that it is able to take advantage of more - modern interfaces to dealing with large sockets, rather than just - select. Perhaps we should look at poll(), Windows completion ports, - and some of the advanced Linux APIs. Select() limits us to - descriptors no higher than FD_SETSIZE, and it may not performa all - that well. We should do some benchmarking and decide on the - interface to use for each platform. May want to take a look at - libevent (http://www.monkey.org/~provos/libevent/) for inspiration. - The libevent home page has some interesting benchmark graphs too. - [Josh implemented poll as a SoC student, but it had problems with - Nsock's architecture. O(1) lookups were becoming O(n) because of - the nature of the data structures. It was slower in his benchmarks. - Nsock would have change from a model of "loop over the event list, - and check to see if the fd for each event is set," to one of "loop - over the fd list, and see if there is a corresponding event for - each. It is the "see if the fd is set" operation that's O(1) with - select (it's FD_ISSET) and O(n) with poll (it's a traversal of a - linked list).] - o Henri added nsock-engines - -o Consider an update feed system for Nmap which let's people obtain - the latest Nmap data files, such as NSE scripts/libs, nmap-os-db, - nmap-service-probes, etc. - o Note that some scripts require updated compiled libraries. We - will need some sort of compatability system. - o One approach is "svn up". Note that Metasploit uses that approach - even for Windows by shipping .svn directories and an svn executable - with the Windows installer. In taht case we might need to have a - separate branch for each release that gets updated version/OS - databases and scripts. - o Another approach is a special feed system as is used by Nessus and - OpenVAS. OpenVAS uses a script wrapper around rsync, or an HTTP - download if that fails. - o Colin's analysis of different methods: - http://seclists.org/nmap-dev/2011/q2/821 - -o [NSE] Consider using .idl files rather than manually coding all the - MSRPC stuff. The current idea, if we do this, is to have an - application in nmap-private-dev which converts .idl files to LUA - code for nmap/nselib. Consider adapting the pidl utility from Samba. - o Drazen did some work on this during SoC. - https://svn.nmap.org/nmap-exp/drazen/nmap-msrpc could get someone - started. - o We moved this out of the active section of the TODO because, while - it is still a good idea and we'd welcome the change if someone wants - to take it on, it isn't something that we are likely to make - progress on unless someone steps forward. - -o Implement a solution for people who want NIST CPE OS detection - results (we'll save version detection for a 2nd phase). Notes: - David report on CPE for OS Detection: - http://seclists.org/nmap-dev/2010/q3/278 - David report on CPE for version detection: - http://seclists.org/nmap-dev/2010/q3/303 - Nessus has described their integration of CPE: - http://blog.tenablesecurity.com/2010/05/common-platform-enumeration-cpe-with-nessus.html. - Older messages about it: - http://seclists.org/nmap-dev/2008/q4/627 - http://seclists.org/nmap-dev/2010/q2/788 - -o [NSE] HTTP spidering library/script - -o We should probably modify stdnse.get_script_args so that it first - checks [scriptname].[argname] and then (if that fails) looks for - [argname] by itself. This way people who are only running one - script or who want to use the same value for multiple scripts that - take the same argument can just give [argname]. But those who want - an argument to only apply to a specific script can give - [scriptname].[argname]. - o The code is in place now, we just need to document the feature. - -o Script review - o Martin Swende patch to force script run - http://seclists.org/nmap-dev/2010/q4/567 - o applied - o irc-info patch. http://seclists.org/nmap-dev/2011/q2/289. - o applied - o http-slowloris. http://seclists.org/nmap-dev/2011/q1/916. - o Had some issues--never got to a state ready for integration - o http-phpself-xss - - Would need to be rewritten to use newer spider.lua. Added an item - to incoming section of Nmap Script Ideas secwiki page. - -o Make new SecTools.Org site with the 2010 survey results. - -o Collect many more IPv6 OS detection training samples from users - - Can start with nmap-dev, but will probably have to do an Nmap - release too. - -o Integrate more NSE scripts, I think our review queue is getting - pretty long. - -o Decide what to do with Henri's nsock-engines branch - (/nmap-exp/henri/nsock-engines). - -o finish making nmap-update part of the nmap windows compile-time - infrastructure - o See if we can build just one project within a solution, rather - than having special "with nmap-update" configuration. - -o Add homedir support to Nmap for the updater - -o Fix expiration date parsing on Nmap Windows for the updater - -o Updater: Make a missing nmap-update.conf nonfatal (perhaps doesn't - even need to mention it). - -o Updater: Clean up the output messages (e.g. only print what user needs to see - unless debugging is specified) - -o [Nping] The --safe-payloads option should be default (though we - should keep it for backward compatability). We could then introduce - --include-payloads for cases where they are desired. - -o A program to canonicalize and tidy nmap-service-probes. - o Order of fields: m p v i d o h cpe:/a cpe:/h cpe:/o. - o Check for duplicate templates (except cpe:). - o Check for unknown templates. - o Canonicalize delimiters (use // first, otherwise try in order - | % = @ #). - o Retain line breaks and comments. - -o Document IPv6 OS detection at https://nmap.org/book/osdetect.html - -o Script review: - - New scripts from Paulino: http-wordpress-brute and http-joomla-brute, - http-majordomo2-dir-traversal.nse, http-trace, http-waf-detect - - http-methods patch. http://seclists.org/nmap-dev/2011/q1/936. - - quake3-info. http://seclists.org/nmap-dev/2011/q2/172. - - smb-os-discovery additional - information. http://seclists.org/nmap-dev/2011/q2/276. - - Outlook web - address. http://seclists.org/nmap-dev/2011/q2/296. [probably not - going to merge to Nmap trunk at this point, though it is good that - the script is available for d/l for those who need it. ] - -o Fix reported (by many people) crash when trying to launch Zenmap on - Mac OS X 10.7 (Lion). - -o Unless we get good arguments for keeping it, we should remove Mac OS - X PowerPC support from our binaries. Apple stopped selling PowerPC - machines in 2006 and they stopped making new OS releases available - for PowerPC as of Snow Leopard (10.6) in August 2009. See this - thread: http://seclists.org/nmap-dev/2011/q3/430 - -o Improvements to the Nmap multicast IPv6 host discovery scripts - - Note that we hope to move them into core Nmap at some point, but - would be good to improve them for now. - - They should probably print the discovered IPv6 addresses, otherwise - they don't actually give the user any information (despite doing - their work) unless you give the newtargets script arg. This would - be similar to the current behavior of broadcast-ping. - - It might be nice if they gave the target MAC address and vendor - when printing the discovered IPv6 information too. Daniel Miller - wrote an initial patch for this (though we need to make sure it can - handle (e.g. doesn't crash for) non-ethernet - devices:http://seclists.org/nmap-dev/2011/q3/862. Our broadcast-ping script - currently prints MAC addresses. - - It is great that the scripts properly use a specific device when - given the Nmap -e option, but they shouldn't require this. They - should do something smart if no specific device name is given. - Examples include performing on all compatable devices or trying to - pick the best device. The all-devices appraoch may be the best, - IMHO. That is how our broadcast-ping script works now. - -o Add anti-spam defenses to secwiki.com to stop the current onslaught - of spam. An extention like ConfirmEdit - (http://www.mediawiki.org/wiki/Extension:ConfirmEdit) may be a good choice. - -o Collect a bunch of IPv6 OS detection signatures from users, - integrate them, and then when we have enough, re-enable OS detection - results. - -o IPv6 OS detection working (when run on) Solaris and AIX - - AIX 6.1 - iSeries / System p - - AIX 7.1 - iSeries / System p - - Solaris 10 - SPARC - -o We should consider splitting a 'brute' category out of the 'auth' - category now that we have so many brute force scripts. I suppose - users can already do "--script *-brute", but having its own category - might still be nice. - -o IPv6 OS detection merge - o [DONE] Initial branch working (nmap-exp/luis/nmap-os6) - o [DONE] Implement the 2 remaining probes - o [DONE] Disable the printing of matches (except maybe with debug on). We - want more training examples first so that results are better. - o [DONE] Merge to /nmap - -o Document Nmap CPE support in appropriate places (candidates: - refguide, os detection book chapter, version detection book chapter, - output book chapter). - -o Finish CPE support code - - Escape certain values that can be inserted into cpe string through - substitution, like cpe:/a:apache:httpd:$1 where $1 contains a - colon. - -o Add advanced IPv6 host discovery features - o Initially done using NSE by adding these scripts: - targets-ipv6-multicast-slaac, targets-ipv6-multicast-invalid-dst, and - targets-ipv6-multicast-echo - -o Initial IPv6 OS detection system (may not make it into stable - though, but we want to at least have it working in a branch first.) - - OK, it is working in nmap-exp/luis/nmap-os6 - -o Investigate a probe/response matching problem reported by QA Cafe - Matthew Stickney and Joe McEachern of QA Cafe. See this thread: - http://seclists.org/nmap-dev/2011/q3/227 - -o When our winpcap installer is run in silent mode - (e.g. "winpcap-nmap-4.12.exe /S"), it seems to execute nmap.exe if - that binary exists in the same directory. This leads to a cmd.exe - window briefly poping up as Nmap displays its console help output. - Moving the Winpcap installer into its own subdir and running it from - there seems to fix this (because it then can't find nmap.exe to - run), but it would be better to determine why this is happening in - the first place and fix it. - -o Obtain Nmap data directory information from nmaprc at runtime rather than - compiled in -- among other advantages this is needed to make - relocateable rpm. [actually we ended up doing this without needing - nmaprc for now] - -o Summer of Code feature creeper: - o Ncat should probably have an --append-output option like Nmap does - so that we can use -o without clobbering existing file. This would - at least be useful for chat.nmap.org. - o Change Zenmap bug reporter so that instead of an automatic - submission system, we print a stack trace and request that the user - send a bug report to nmap-dev. - -o [Ncat] Solve a crash that only happens on Windows when connecting - with --ssl-verify and -vvv, for example - ncat --ssl-verify -vvv www.amazon.com 443 - The crash happens in the function verify_callback, when the function - X509_NAME_print_ex_fp is called. Just commenting those two calls - avoids the problem. By trying different combinations of debug print - statements, I once got the message - OPENSSL_Uplink(10109000,08): no OPENSSL_Applink - This refers to a Windows dynamic linking issue: - http://www.openssl.org/support/faq.html#PROG2 - However I tried both including and changing the - linker mode to /MD, and neither changed the behavior. - Changing the flags from XN_FLAG_ONELINE to 0 seems to make the - problem go away. - -o Integrate new OS detection submissions (We have about 1,700 - submissions since 11/30/10) - -o Nmap should defer address parsing in arguments until it has read - through all the args. Otherwise you get an error if you use like -S - with an IPv6 address before you put -6 in the command line. You get - a similar problem if you do "-A -6" (but "-6 -A works properly). - This is a possible feature creeper task. - -o Ncat chat (at least in ssl mode) no longer gives the banner greeting - when I connect. This worked in r23918, but not in r24185, which is - the one running on chat.nmap.org as of 6/20/11. Verify by running - "ncat --ssl -v chat.nmap.org" - -o IPv6 Neighbor Discovery-based host discovery (analog to ARP scan). - -o Investigate and document how easy it is to drop Ncat.exe by itself - on other systems and have it work. We should also look into the - dependencies of Nmap and Zenmap. It may be instructive to look at - "Portable Firefox" - (http://portableapps.com/apps/internet/firefox_portable) which is - built using open source technology from portableapps.com, or look at - "The Network Toolkit" by Cace - (http://www.cacetech.com/products/network_toolkit.html). For Nmap - and Nping, we may want to improve our Winpcap to load as a DLL - without requiring installation. There is a separate TODO item for that. - -o The SCRIPT_NAME variable should not include the ".nse" in script - names. Currently, it omits that for scripts in the DB, but includes - it for scripts you specify based on their filename. See: - http://seclists.org/nmap-dev/2011/q2/481 - -o If possible, Ncat, in listen mode, should probably listen on the system's - IPv6 interfaces as well as IPv4. This is what servers like apache - and ssh do by default. It might now be possible to listen on IPv6 - by running a second ncat with -6, but that doesn't really work for - broker and chat modes because you want the IPv6 users to be able to - talk to IPv4 and vice versa. - - This was partially implemented, but still doesn't seem to work in - --chat mode. Can test against chat.nmap.org - - Done. Tested on scanme with David & Fyodor on 7/18/11. - -o Right before the release, we could build Ncat portable and post it - on https://nmap.org/ncat/. - - Actually we did that for 5.59BETA1, which is good enough for now. - -o CHANGELOG updates [Fyodor] - -o [Ncat] Add new certificate bundle (ca-bundle.crt) since the current - one is out of date. See http://seclists.org/nmap-dev/2011/q2/641. - -o Move these prerule/postrule script ideas to secwiki script idea page - if appropriate (with a bit more details): - o AS Number to IP ranges: http://seclists.org/nmap-dev/2010/q2/101 - In progress. - o DNS service discovery (Bonjour): http://en.wikipedia.org/wiki/Bonjour_%28software%29 - Present as dns-service-discovery.nse. - o Netbios Name Service - Already present as broadcast-netbios-master-browser.nse? - o DHCP broadcast requests - Present as dhcp-discover.nse. - o Postrules could be created which give final reports/statistics or - other useful output. Like a reverse-index, which shows all the open - port numbers individually and the hosts which had that port open - (e.g. so you can see all the ssh servers at once, etc.) - Admittedly you can do that pretty easy with Zenmap instead. - Have a few of these: ssh-hostkey and upcoming creds-summary. - o We could have a prerule sniffer script which uses pcap to sniff - traffic for some short configurable amount of time and then adds the - discovered hosts to the target list. - Already present as targets-sniffer.nse. - o We could have a script which takes traceroute results and adds them to the target list. - Already present as targets-traceroute.nse. - -o [NSE] Add these ideas to secwiki script ideas page if appropriate - (with a bit more details): - o Windows system logs (like sysinternals' psloglist) - o Services (like sysinternals' psservice) - o A script (or modification to smb-check-vulns) to - detect this MSRPC vulnerability: - http://seclists.org/fulldisclosure/2010/Aug/122 - o BasicHTML/XML parser library? For example, Sven Klemm wrote a script - which uses libxml2: http://seclists.org/nmap-dev/2008/q3/0462.html. - And here is one by Duart Silva using Expat: - http://seclists.org/nmap-dev/2009/q3/1093. - o Add detection of duplicate machines via IP.ID technique. - Maybe I should use uptime timestamps too. Oh, and MAC addresses - too. Our SSH host key script is useful for this as well. - -o Look into iplog ( http://ojnk.sourceforge.net/ ) -z option which is - supposed to fool OS detection. - o The software is no longer maintained, so we're not going to worry - about it. The page says: "I am through working on this project. I - will not be making any updates, and I will ignore just about all - email about it. If anybody wants to take it over (for whatever - reason), let me know" - -o [NSE] Consider how we compare to the Nessus Web Application Attack - scripts - (http://blog.tenablesecurity.com/2009/06/enhanced-web-application-attacks-added-to-nessus.html). - [Joao making a list of web scripts which we might find useful, - Fyodor asking HD moore for permission to use http enum dir list] - -o [NSE] HTTP persistant connections/keepalive? May make - spidering/grinding/auth cracking more efficient - -o [NSE] HTTP Pipelining support? May make spidering/grinding/auth - cracking more efficient - -o [NSE] HTTP Cookie suppport? Might be useful for spidering sites which use it - for authentication/authorization/personalization. - -o [NSE] URL grinder checks for existence of applications in common/default - paths. Scanning http paths to see if they exist is in some ways - similar to scanning to see which ports are open. - o Our http-enum does this. - -o Investigate why and whether we need mswin32/pcap-include/pcap-int.h. - This file is not included in the official WinPcap 4.1.1 developers' - pack - (http://www.winpcap.org/install/bin/WpdPack_4_1_1.zip). Presumably - it covers internal functions and structures which we aren't really - supposed to access it. If we can get rid of it, that would be - great. If we need it, we should probably upgrade to the - 4.1.1. version (presumably from the Winpcap source code - distribution). Right now it is included in tcpip.h, - nsock/src/nsock_pcap.h, and nping/common_modified.cc: o David looked - into it. He says it isn't distributed with the WinPcap developer's - pack. You have to extract it from the source file. He updated to the - 4.1.1 version. He says The entire reason we need it is so we can - peek at the definition of struct pcap, so we can access the - pcap.adapter member on Windows. In order to pass it to - PacketSetReadTimeout. Usually struct pcap is an opaque type and you - are only supposed to access it through a pcap_t *. Unfortunately I - don't think there's an easy way to manipulate the timeouts in - WInPcap like we do on other platforms. You can specify a timeout - when you do pcap_open, but we like to set a timeout on every - read. So we sort of sneak in and call PacketSetReadTimeout. In the - code there's even a comment: "BUGBUG: This is cheating." libdnet - also uses the Packet* functions, but in a more innocuous - way. It doesn't access them through a struct pcap, so it - doesn't need pcap-int.h. David tried testing whether this makes - any signficiant difference--to see if we could just remove the - PcapSetReadTimeout()--but that didn't work out. - - We're not going to worry about this for now since it isn't - important enough to pester the pcap people about, and they don't - seem to be changing their internal structure anyway. And if they - do, we can get the new pcap-int.h. - -o Further brainstorm and consider implementing more prerule/postrule - scripts: - o [Implemented] dns-zone-transfer - o [Implemented, but a joke] http-california-plates - -o Investigate this interface-matching problem on Windows: - http://seclists.org/nmap-dev/2011/q1/52. It is related to the - libdnet changes we made to allow choosing the correct physical - interface when teamed interfaces share the same MAC. - I think this is solved with the rewritten libdnet code (that uses - GetAdaptersAddresses) in my nmap-ipv6 branch. --David - -o [Ncat] When in connection brokering or chat mode with ssl support - enabled, if one client connects and doesn't complete ssl negotiation, - it hangs any other connections while that first is active. One way to - reproduce: - Run SSL chat server like: /usr/local/bin/ncat --ssl -l --chat - Window #1: Connect without ssl: ncat -v chatserverip - Window #2: Try to connect with SSL: ncat -v --ssl chatserverip - Window #2 will not work while #1 is active. If you quit #1, #2 - should work again. - -o IPv6 todo. - - Protocol scan (-sO). - -o [Ncat] Find out what RDP port forwarding apparently doesn't work on - Windows. http://seclists.org/nmap-dev/2011/q1/86 - -o Add raw packet IPv6 support, initially for SYN scan - o After that can add UDP scan, and sometime OS detection (David did - some research on what IPv6 OS detection might require). - -o When I (Fyodor) scan scanme.nmap.org with the command "nmap -sC -p80 --Pn -n scanme.nmap.org", I get a blank http-favicon line like: - 80/tcp open http - |_http-title: Go ahead and ScanMe! - |_http-favicon: - But if I use "--script http-favicon" instead of -sC, it works fine. - -o UDP scanning with IP options causes "Received short ICMP packet" on - receipt. http://seclists.org/nmap-dev/2011/q1/82 - - -o [Zenmap] Make formerly open ports that are now closed or filtered - disappear from the "Ports / Hosts" tab. This appears to be related - to ignored states; if in the second scan I use -d2 so all ports are - included in the output, the interface is updated correctly. - http://seclists.org/nmap-dev/2010/q4/659 - -o [Zenmap] When a target is unresponsive (and its distance isn't - known), put it at the next furthest ring from the known traceroute - hosts (with a dashed line), instead of putting it at the first ring. - See http://seclists.org/nmap-dev/2011/q1/834. - -o Rewrite the portreasons code not to use parallel arrays - (reason_text, reason_pl_text) and not to require special alignment - between the enum codes and (for example) ICMP types. Instead define - one structure containing all relevant information about a reason, - and define helper functions to map ICMP types to reason codes. In - particular, code like this needs to go away: current_reason = - ping->type + ER_ICMPTYPE_MOD; if (current_reason == ER_DESTUNREACH) - current_reason = ping->code + ER_ICMPCODE_MOD; - -o Fix memory consumption problem in drda-info (see - http://seclists.org/nmap-dev/2011/q2/451) - - Fixed (turned out to affect a lot of scripts) - -o Script dispensation - - sip-enum-users and - sip-brute. http://seclists.org/nmap-dev/2011/q2/56. - o Merged - - xmpp. http://seclists.org/nmap-dev/2011/q2/239. - o Merged - -o Script review/disposition: - - Merged: DNSSEC enumeration. http://seclists.org/nmap-dev/2011/q1/406. - - Merged: quake3-master-getservers patch. http://seclists.org/nmap-dev/2011/q1/925. - - Merged: backorifice-info. http://seclists.org/nmap-dev/2011/q2/185. - - Merged: omp2-brute and omp2-enum-targets. http://seclists.org/nmap-dev/2011/q2/231. - - Merged: http-wp-plugins. http://seclists.org/nmap-dev/2011/q1/806. - -o Decide what to do about ms-sql-info slowing scans: - http://seclists.org/nmap-dev/2011/q1/913 - - patch applied: http://seclists.org/nmap-dev/2011/q1/1102 - -o Script disposition - - Patch to get interfaces by Djalal. - http://seclists.org/nmap-dev/2011/q1/291 - - Incorporated - - epmd-info. http://seclists.org/nmap-dev/2011/q1/931. - - Incorporated - - google-id. http://seclists.org/nmap-dev/2011/q1/952. - - Incorporated as http-affiliate-id - -o [Ndiff] should, in non-verbose mode, perhaps not print the changed - Nmap version and/or scan time if nothing else has changed between - two files. See http://seclists.org/nmap-dev/2011/q1/674. - -o Script review disposition: - - ssl-known_key http://seclists.org/nmap-dev/2010/q4/733 - Thread continues at http://seclists.org/nmap-dev/2011/q1/26. - - Merged - - dns-nsec-enum - - Merged - -o The file /nmap/mswin32/icon1.ico is used by the NSIS installer to - set the Nmap uninstall icon (I'm not sure if it is used for anything - else). But this is a very old icon and doesn't match the blue eye - we use now. So we should probably update that with a modern "blue - insecure eye" icon. I (Fyodor) tried simply replacing icon1.ico - with http://insecure.org/shared/images/tiny-eyeicon.ico, but that - didn't work. It must not meet the required format. - -o Add some content to https://secwiki.org and announce it. - -o Removing -sR option (but keeping the functionality as part - of -sV). See http://seclists.org/nmap-dev/2011/q1/688 - - Update Nmap documentation/book to remove it there too - - -o Script disposition: - - dns-brute by cirrus. http://seclists.org/nmap-dev/2011/q1/351 - Should share domain list with http-vhosts. - git://code.0x0lab.org/nmap-dns-brute.git - - Added by David - -o Write and post 2010 SoC Successes writeup [Fyodor] - -o Script review - - quake3-master-getservers http://seclists.org/nmap-dev/2011/q1/64 - [merged] - - dpap-brute by Patrik Karlsson. - http://seclists.org/nmap-dev/2011/q1/252. - [merged] - -o The -V option to Nmap, in addition to reporting the version number, - should give details on how Nmap was compiled and the environment it - is running on. This includes things like whether SSL is enabled, - the platform string, versions of libraries it is linked to, and - other stuff which is often useful in debugging problems. - o We want to list at least: - o Nmap version number (that line is fine as is) - o host platform string (for which it was compiled) - o Whether OpenSSL and LibSSL, NLS, and IPv6 are enabled - - Version number of OpenSSL and LibSSL if those are enabled - o Version numbers of libdnet, libpcre, and libpcap - -o Script review: - - SCADA scripts http://seclists.org/nmap-dev/2010/q4/612 - http://seclists.org/nmap-dev/2010/q4/613 - http://seclists.org/nmap-dev/2010/q4/623 - http://seclists.org/nmap-dev/2010/q4/639 - [on hold] - - servicetags http://seclists.org/nmap-dev/2010/q4/691 - needs new testing on OpenSolaris: http://seclists.org/nmap-dev/2011/q1/91 - [committed] - - firewalk-path http://seclists.org/nmap-dev/2011/q1/63 - [committed over previous firewalk script] - - snmp-ios-config http://seclists.org/nmap-dev/2011/q1/10 - Requires a TFTP server; decision was to build such server in Lua - if possible. Patrik Karlsson's beginning TFTP implementation: - http://seclists.org/nmap-dev/2011/q1/169. - [committed by Patrik] - -o Script merged: p2p-dropbox-listener - http://seclists.org/nmap-dev/2010/q4/689 - -o A trivial change: we currently print some lines about NSE - pre-scanning and post-scanning in verbose mode even when no such - scripts are being run. We should not print those in that case. For - example, nmap -A -v scanme.nmap.org gives me these superfluous lines: - NSE: Script Pre-scanning. - NSE: Starting runlevel 1 (of 2) scan. - Initiating NSE at 12:23 - Completed NSE at 12:23, 0.00s elapsed - NSE: Starting runlevel 2 (of 2) scan. - NSE: Script scanning 64.13.134.52. - NSE: Starting runlevel 1 (of 2) scan. - Initiating NSE at 12:24 - Completed NSE at 12:24, 4.14s elapsed - NSE: Starting runlevel 2 (of 2) scan. - NSE: Script Post-scanning. - NSE: Starting runlevel 1 (of 2) scan. - NSE: Starting runlevel 2 (of 2) scan. - -o Do new Nmap release with the stuff merged from SoC students and - other new developments. - -o Modify Zenmap to use the new --script-help system to enumerate - scripts and collect information such as their descriptions. This - will resolve the problem of Nmap's broadcast prerule scripts running - when you open the profile editor. - -o Document --script-help in docs/refguide.xml and docs/scripting.xml. - -o [Zenmap] Brian Krebs found a problem (which Fyodor is able to - reproduce) in the target selector on the left pane. When you select - one of the scanned targets, it is supposed to jump to that target in - the "Nmap Output" tab on the right pane. Instead, nothing seems to - happen. One of our output format changes probably broke the - feature. It still works fine if you have the "Ports / Hosts" or - "Host Details" tabs active in the right pane instead. - -o Include a --script-help system to Nmap, which provides user readable - text help and also machine parsable XML information for scripts - which match a pattern (e.g. the same sort of arguments you could use - for --script, like a category or http-* or whatever). The - --script-help ONLY provides help and quits, it does not run the - script. For some initial implementation work, see this thread: - http://seclists.org/nmap-dev/2011/q1/163 - -o [Nping] See whether --echo-client mode really requires root, and - remove that restriction if not. - Luis explanation for requiring root: - http://seclists.org/nmap-dev/2011/q1/248 - -o Script review: - - p2p-dropbox-listener http://seclists.org/nmap-dev/2010/q4/689 - -o Decide whether to include NSE console script help, decide on - implementation issues. http://seclists.org/nmap-dev/2011/q1/163 - -o [Zenmap] Use a more efficient algorithm to update the display of Nmap normal - output in live scans. - zenmapGUI.NmapOutputViewer.NmapOutputViewer.refresh_output calls - zenmapCore.NmapCommand.NmapCommand.get_output, which re-reads the - entire output file (into memory) and then puts it in the text buffer - if it has changed. So already we're storing the whole output twice in - memory. When the text field changes, update_output_colors - re-highlights the whole file. - -o Update changelog to note recent changes - -o Do final dev/test release - -o If Nping is compiled w/o SSL support, and the user specifies an - encryption key, it should fail and insist they use --no-crypto - rather than ignoring the key and omitting crypto. Otherwise the - user might think they're getting encryption when they're not. David - found this problem in the server, and we also should check how the - client behaves. - -o [Ncat] Make --exec work in conjunction with --proxy. The --proxy - code path skips the --exec code. See - http://seclists.org/nmap-dev/2010/q4/604 and the test "--exec - through proxy" in ncat-test.pl. - -o Decide what to do about Nmap static binaries failing to work on new - Fedora releases (and others?). See these threads: - http://seclists.org/nmap-dev/2011/q1/46 and - http://seclists.org/nmap-dev/2010/q1/308 - o We ended up dynamically linking system libs in the RPM rather than - statically linking them. We still statically link things like lua, - pcre, ssl, etc. - -o Fix our mac builds so that they contain SSL support again (5.35DC1 - did, but TEST1 and TEST2 didn't for some reason. - -o Add our broadcast discovery scripts to a "broadcast" category (they - should generally just be in "broadcast" and (assuming they are safe) - "safe", and not normal "discovery". Update scripting.xml to note - this new category too. - -o The latest IANA services file - (http://www.iana.org/assignments/port-numbers) has many identified - services which are still "unknown" in our files because ours is - based on a much older version of that file. We should probably take - that file and add names and comments to our nmap-services-all where - they are "unknown" in our file. An example of such a port is 3872, - oem-agent. - -o Script review: - - patch for ftp-proftpd-backdoor - http://seclists.org/nmap-dev/2010/q4/678 - - patch for hddtemp-info http://seclists.org/nmap-dev/2010/q4/676 - -o We should probably update our Windows build systems to use Python - 2.7. As of 11/8, it looks like all our dependency libraries are - available for 2.7: - o David upgraded and it worked, though Rob found a potential problem - and added vcredist 2008. Fyodor will test on the official Win7 Nmap - build system. - PyGTK: 2.22.0 IS available for 2.7 - PyCairo: 1.8.10 IS available for 2.7 - PyGObject: 2.26.0 IS available for 2.7 - Py2exe: 0.6.9 IS available for 2.7 - -o Do service/version detection submission integration (last done in - April) - -o Do os detection submission integration (last done in April) - -o Script review: - - modbus-enum http://seclists.org/nmap-dev/2010/q4/489 - -o Create Nmap wiki - o Decide on domain name - o Include insecure Chrome - o Decide on wiki software, probably just use mediawiki - o install it on a Linode, probably Web - -o [NSE] Web application fingerprinting script. Would be great to be - able to take a URL and determine things like "this is Joomla" or - "this is Plone" or "Mediawiki" or whatever. Rather than hard code - regular expressions or other tests in a script, it should use a - signature file like Nmap OS and version detection do. Might work in - combination with URL grinder to check for applications at - default/common locations. See also a script that does favicon - scanning TODO item. - - http-enum pretty much does this now. - -o Update our distribution build systems and documentation to use - Visual C++ 2010 Express rather than the 2008 version. See - http://www.microsoft.com/express/Windows/ - -o Dependency licensing issues (OpenSSL, Python, GTK+, etc.) - o Almost done! We just have some file renaming/organizing left to do. - o We should do an audit to ensure that we are in complete compliance for the - licenses of all the software we ship in any of our downloads, as some - licenses have special clauses for things like including their - license/copyright file, mentioning them in our documentation, etc. - And of course we want to credit them properly even where the license - doesn't require it. We should probably make a list of these in our - docs/ directory along with any special information/requirements of - their license. And maybe we should put the current licenses in a - subdir too. In particular, these come to mind: - o libpcre - o lua - o OpenSSL - o libpcap - o GTK+/Glib/ATK/Pango/PyGTK (Win/Mac versions of Zenmap link to - PyGTK) - o SQLite - o Python (Win/Mac versions of Zenmap link to Python) - o X.org libraries (Mac version links to them) - o libdnet - -o Small NSEDoc bug: - https://nmap.org/nsedoc/scripts/dns-zone-transfer.html contains 'id - \222\173' near the bottom. This is presumably due to misparsing this - line from the script: local req_id = '\222\173'. Given that we don't - use IDs any more, maybe we can just get rid of the functionality. - -o [NSE] We should probably enable broadcast scripts to work better by - (initial thoughts): - o Done and merged by David! - 1) Change NSE to always set nsp_setbroadcast() on new sockets - 2) Change nsock to create real sockets at time of nsi_new so you can - bind to them. - See this thread (only some of the messages involve broadcast - support): http://seclists.org/nmap-dev/2010/q3/357 - -o [NSE] Review scripts: - o Hostmap (Ange Gutek) - http://seclists.org/nmap-dev/2010/q3/159 - -o Post BH/Defcon Nmap videos - -o Let Nsock log to stderr, so its messages don't get mixed up with the - output stream when Ncat is run with -vvv. - http://seclists.org/nmap-dev/2010/q3/113 - -o [NSE] Our http-brute should probably support form POST method rather - than just GET because some forms require that. - -o Nping needs to call nsp_delete so that its socket descriptors are - not left behind. - -o [Zenmap] Add a button to select script files from the filesystem. - -o [Zenmap] Show help for individual script arguments in the Help pane, - not for all arguments at once. - -o Upgrade our Windows OpenSSL binaries from version 0.9.8j to the - newest version (1.0.0a as of Aug 12, 2010). - -o Since Libdnet files (such as ltmain.sh) are apparently only used by - libdnet (they used to be used by shared library NSE C scripts), we - should move them to the libdnet directory. - o Turned out to be a pain. See - http://seclists.org/nmap-dev/2010/q3/733 - -o [Zenmap] Consider a memory usage audit. This thread includes a claim - that a 4,094 host scan can take up 800MB+ of memory in Zenmap: - http://seclists.org/nmap-dev/2010/q1/1127 - The reporter mentioned Guppy/Heapy to debug memory use: - http://guppy-pe.sourceforge.net/ - http://www.pkgcore.org/trac/pkgcore/doc/dev-notes/heapy.rst. Many - Nmap survey respondants complained about this too. - Note: Fyodor has a 50MB scan log file named ms-vscan.xml which - demonstrates this problem. When trying to load the file, Zenmap - grows to 1150MB of RAM, pegs the CPU usage at 100% for many - minutes or maybe hours (I forgot about it, but woke up the next day - to find that it had started, was then using 2.4GB of RAM. The - hosts/services functionality seemed to work, although it would take - a minute or so to switch from say "ftp" port to view "ssh" ports. - -o [NSE] Maybe we should create a script which checks once a day - whether similar tools (Metasploit, Nessus, OpenVAS, etc.) have any - new modules, and then mails out a list of them with the description - fields. The mail could go to just interested parties, or maybe - nmap-dev. This may help prevent important vulnerabilities from - falling through the cracks. Perhaps we would include new NSEs in - there too, especially if we open it up as a public list. - -o Now that NSE has more script phases (prerule, postrule, hostrule, - portrule, and versionrule soon to come), the NSEDoc should specify - which phases a script belongs to. - -o Consider implementing a nsock_pcap_close() function or making - nsp_delete() call pcap_close() when pcap IODs are used. Currently valgrind - warns about a socket descriptor left opened (at least in Nping). - See http://seclists.org/nmap-dev/2010/q3/305. - o It turns out that the pcap descriptors are being closed properly, - but Nping isn't calling nsp_delete. - -o [NSE] High speed brute force HTTP authentication. Possibly POST and - GET/HEAD brute force cracking. [done except for form POST, adding - separate TODO item for that] - -o [NSE] Review scripts: - o New brute, vnc, and svn scripts by Patrik. This guy is a coding - machine :). http://seclists.org/nmap-dev/2010/q3/111 - o rmi-dumpregistry by Martin - Swende. http://seclists.org/nmap-dev/2010/q2/904 - o path-mtu.nse - http://seclists.org/nmap-dev/2010/q3/222 - o 15 more from Patrik :). http://seclists.org/nmap-dev/2010/q3/284 - -o [NSE] Consider modifying our brute force scripts to take advantage - of the new NSE multiple-thread parallelism features. - - We've done this with db2-brute, but the DB may have been a - bottleneck there, so we should probably do more testing after - modifying another script for this sort of parallel cracking. - -o Look into implementing security technologies such as DEP and ASLR on - Windows: http://seclists.org/nmap-dev/2010/q3/12. - -o Ncat and Nmap should probably support SSL Server Name Indication - (SNI). See this thread: http://seclists.org/nmap-dev/2010/q3/112. - We need this to talk to web servers which share one SSL IP and port - because we need to ask for the right SSL key. - -o [NSE] In the same way as our -brute scripts limit their runtime by - default, I think qscan should be less intense by default. For - example, perhaps it could run by default on no more than 8 open - ports, plus up to 1 closed port. Right now it does things like - running on 65,000+ closed ports and bloats scan time (and output). - Of course there could (probably should) still be options to enable - more intense qscanning. - -o [Web] We should see if we can easily put the Insecure chrome around - Apache directory listings and 404 pages (e.g. https://nmap.org/dist/ - and https://nmap.org/404). I think we may have had this working - before the move to Linode, so maybe check conf/httpd.conf.syn. - -o Do a serious analysis if and how we should use the NIST CPE standard - (http://cpe.mitre.org/) for OS detection and (maybe in a different - phase) version detection results. One thing to note is that they - may not have entries for many vendors we have. For example, one - person told me they couldn't find SonicWall or D-Link in the CPE - dictionary. Here are some - discussions threads on adding CPE to Nmap: - http://seclists.org/nmap-dev/2008/q4/627 and - http://seclists.org/nmap-dev/2010/q2/788. - Nessus has described their integration of CPE at - http://blog.tenablesecurity.com/2010/05/common-platform-enumeration-cpe-with-nessus.html. - -o [NSE] Create NSE scripts to scan for and/or exploit these VXWorks issues: - http://blog.metasploit.com/2010/08/vxworks-vulnerabilities.html [Ron - may be able to do this. Or others are welcome to take a shot at it.] - -o The -g (set source port) option doesn't seem to be working (at least - in Fyodor's quick tests) for version detection or connect() scan, - and apparently doesn't work for NSE either. We should fix this - where we can, and document the limitation in the refguide where it - is impractical. Also see http://seclists.org/nmap-dev/2010/q2/576. - -o [Zenmap] script selection interface for deciding which NSE scripts to - run. Ideally it would have a great, intuitive UI, the smarts to - know the scripts/categories available, display NSEdoc info, and even - know what arguments each can take. - -o Review http-xst (Eduardo Garcia Melia) - - http://seclists.org/nmap-dev/2010/q3/159 - -o [NSE] Investigate sslv2.nse falsely reporting SSLv2 as being - supported. - http://seclists.org/nmap-dev/2010/q2/754 - -o [NSE] The NSEDoc for some scripts includes large "Functions" - sections which aren't really useful to script users. For example, - see https://nmap.org/nsedoc/scripts/snmp-interfaces.html. Perhaps we - should hide these behind an expander like "Developer documentation - (show)". I don't think we need to do this for libraries, since - developers are the primary audience for those documents. - o Talked to David. We should just remove the function entries. - -o We should add a shortport.http or similar function because numerous - services use this protocol and many of our scripts already try to - detect http in their portrule in inconsistent ways. - -o [NSE] Maybe we should create a class of scripts which only run one - time per scan, similar to auxiliary modules in Metasploit. We - already have script classes which run once per port and once per - host. For example, the once-per-scan ("network script"?) class might - be useful for broadcast LAN scripts (Ron Bowes, who suggested this - (http://seclists.org/nmap-dev/2010/q1/883) offered to write a - NetBIOS and DHCP broadcast script). Another idea would be an AS to - IP ranges script, as discussed in this thread - http://seclists.org/nmap-dev/2010/q2/101 [Could be a good SoC - infrastructure project] - o David notes: "I regret saying this before I say it, because I'm - imagining implementation difficulties, we should think about - having such auxiliary scripts be able to do things like host - discovery, and then let the following phases work on the list it - discovers." - -o Analyze what sort of work would likely be required for Nmap to - support OS detection over IPv6 to a target. - o Would probably start with a way to send raw IPv6 packets - o There is a raw IPv6 patch here: - http://seclists.org/nmap-dev/2008/q1/458 - o Also it looks like Nping may be doing this already. - o Then we need to figure out if we can use our current DB and - techniques, or if we'd likely thave to have an IPv6-specific - DB. [David] - -o July Nmap releases (at least a beta version, and maybe a stable - too). Last release was 5.30BETA1 on March 29 - -o Add this patch for compilation on OpenSolaris. - http://blogs.sun.com/sdaven/entry/nmap_5_35dc1_compile_on - -o Now that we've put the ndiff, ncat, and nping man pages under the - scope of the book (e.g. https://nmap.org/book/ncat-man.html), we need - to add a redirect from the old locations and also update our links. - -o Make sure the long output lines in Nping's man page are OK for the book. - See r18829 and r18864. - -o Update "History and Future of Nmap" - (https://nmap.org/book/history-future.html) to include all the news - since September 2008. [Fyodor] - -o Fix Win7 networking issue reported by Luis which seems to have been - triggered by r17542. See this thread: - http://seclists.org/nmap-dev/2010/q3/40 - -o Upgrade to WinPcap 4.1.2 - Rob has a patch - See this thread: - http://seclists.org/nmap-dev/2010/q3/18 - -o [NSE] Review UnrealIRCd backdoor detection script - http://seclists.org/nmap-dev/2010/q2/854 - -o [Zenmap] Investigate segfault on some installs of OS X 10.6.3: - http://seclists.org/nmap-dev/2010/q2/587 - o David rebuilt with MacPorts 1.9.1 rather than 1.8.2 and the - problem went away. - -o [Zenmap] Investigate failure to start on some installations of OS X - 10.6.3. - [ We think one may just not have waited long enough as he said it - started working, and another case (the 587) seems to be a - segfault--we added a new task for that ] - http://seclists.org/nmap-dev/2010/q2/587 - http://seclists.org/nmap-dev/2010/q2/859 (He responded to David - privately and said that it was not an I7 processor.) - Nmap seems to be having problems too: - http://seclists.org/nmap-dev/2010/q2/747 - -o [NSE] Review Gutek's PHP version disclosure script. - http://seclists.org/nmap-dev/2010/q2/569 - -o Fix the IPv6 name resolution problem described in this thread: - http://seclists.org/nmap-dev/2010/q2/787 - -o [NSE] Review Gutek's libopie detection/DOS script. - http://seclists.org/nmap-dev/2010/q2/635 - -o [NSE] Review Gutek's web server directory traversal script. - http://seclists.org/nmap-dev/2010/q2/595 - - It became modifications to http-passwd - -o [NSE] Review dns-cache-snoop.nse from Eugene Alexeev. - http://seclists.org/nmap-dev/2010/q2/195 - Better attachment at: http://seclists.org/nmap-dev/2010/q2/200 - Need to decide on a domain list: http://seclists.org/nmap-dev/2010/q2/199 - -o Fix bug where multiple targets with the same IP can end up in a - hostgroup and cause port scanning and probably OS detection to - misbehave. An example is "nmap -F scanme2.nmap.org - scanme3.nmap.org". See this thread for details: - http://seclists.org/nmap-dev/2010/q2/322 - -o Need to fix our current win32.zip distribution so that .svn files - aren't included (currently they are in nselib/data). Will probably - be a simple adjustment to mswin32/Makefile. - -o Make Zenmap splash screen - -o [NSE] Add one of, or combine, ntp-peers and ntp-monlist. - http://seclists.org/nmap-dev/2010/q2/190 - http://seclists.org/nmap-dev/2010/q2/191 - -o [NSE] Reorganize nselib to allow libraries in subdirectories. - Currently, to avoid expanding the number top-level libraries, code - that is only used by one library is built into that library's file, - even if it is logically separate. For example, the mongodb library - contains a BSON-parsing library. Instead, that library could go in - mongodb/bson.lua. The msrpc and smb libraries could potentially be - broken up in this way. - UPDATE: We decided not to do this for now, given complications in - nsedoc, packaging, etc. to support the new hierarchy. Instead, we - can use prefixes like we do with scripts (e.g. mongodb-bson.lua, - msrpc-types.lua). - -o Add a configure option to our libpcap which enables an older Linux - packet capture system (David's noring patch). This is needed in - some cases for 32-bit static binaries to work on 64-bit Linux - systems. Note that it is unneccessary if both the build system and - the target system use Linux 2.6.27, as that has an architecture - independent tpacket_hdr (called tpacket2_hdr). [Added by David as - --disable-packet-ring] - -o Test Jay Fink's UDP payload prototype. - http://seclists.org/nmap-dev/2010/q1/168 - [ tested, improved, merged by David] - -o Resolve Ncat broadcast support issue (see this thread: - http://seclists.org/nmap-dev/2010/q2/422). - -o [NSE] Review and test the DB2 library and - scripts. http://seclists.org/nmap-dev/2010/q2/395 (but updated - versions may be available). - -o Move nmap/docs/TODO into its own todo directory (probably nmap/todo) - and then encourage maintainers of /status/ TODOs and any other TODOs - to migrate theirs there. Unlike the status directory, /nmap/todo - would be readible by anyone. [Fyodor] - -o Nmap should at least print (and maybe scan) all IP addresses for - hostnames specified on the command line. We will start with just - printing all the addresses. Here is a thread on the topic: - http://seclists.org/nmap-dev/2010/q2/302 - [David made it do the printing, adding a different task related to - scanning them all] - -o Integrate new service detection fingerprint submissions (we have - more than 730 since Dec. 17, 2009. - -o [Ncrack] Use our new password lists (now used by NSE) for Ncrack as - well. Ncrack can probably handle a larger list than NSE uses. - -o Consider MSRPC ideas from Ron--we might want to add some as TODO - tasks: http://seclists.org/nmap-dev/2010/q2/389 - -o Fix XML inconsistency described at - http://seclists.org/nmap-dev/2010/q2/326 - -o Integrate new OS fingerprints (we have more than 1,300 since - November 10, 2009). - -o Finish selecting GSoC 2010 projects - -o Upgrade libpcap to the new 1.1.1 version. - -o Improve the NSI installer by adding command-line options for unsetting - each of these GUI checkboxes individually (particularly useful for - silent mode): - LangString DESC_SecCore ${LANG_ENGLISH} "Installs Nmap executable, NSE scripts and Visual C++ 2008 runtime components" - LangString DESC_SecRegisterPath ${LANG_ENGLISH} "Registers Nmap path to System path so you can execute it from any directory" - LangString DESC_SecWinPcap ${LANG_ENGLISH} "Installs WinPcap 4.1 (required for most Nmap scans unless it is already installed)" - LangString DESC_SecPerfRegistryMods ${LANG_ENGLISH} "Modifies Windows registry values to improve TCP connect scan performance. Recommended." - LangString DESC_SecZenmap ${LANG_ENGLISH} "Installs Zenmap, the official Nmap graphical user interface. Recommended." - LangString DESC_SecNcat ${LANG_ENGLISH} "Installs Ncat, Nmap's Netcat replacement." - LangString DESC_SecNdiff ${LANG_ENGLISH} "Installs Ndiff, a tool for comparing Nmap XML files." - LangString DESC_SecNping ${LANG_ENGLISH} "Installs Nping, a packet generation tool." - -o We should have a standard function which takes time arguments in the - same format as Nmap does (e.g. 60s, 1m, etc.) and the scripts which - take time arguments should be modified to use it. David suggests - this here: http://seclists.org/nmap-dev/2010/q2/35. We are also - going to update the normal Nmap timing functions to take seconds by - default, as described here: http://seclists.org/nmap-dev/2010/q2/159 - -o Nmap should probably always produce a well-formed XML file, even if - it exits with a fatal() error. In that case, the error should be - included in the XML. Right now, for example, if the network is - down, the XML output will just stop (no closing tags) and Nmap will - print something to STDERR like: - nexthost: failed to determine route to 9.48.184.164 - QUITTING! - -o Get @output sections for the last remaining scripts w/o them: - [WARN] script auth-spoof missing @output - [WARN] script db2-das-info missing @output - [WARN] script db2-info missing @output - [WARN] script http-passwd missing @output - [WARN] script iax2-version missing @output - [WARN] script ms-sql-config missing @output - [WARN] script ms-sql-query missing @output - [WARN] script oracle-sid-brute missing @output - [WARN] script pop3-brute missing @output - [WARN] script pptp-version missing @output - [WARN] script skypev2-version missing @output - -o [Zenmap] Maybe it should sort IPs in an octet-aware way. And maybe - you should be able to sort by IP address (perhaps that should be the - default). Current plan is to just sort by IP by default, and maybe - we'll offer other sort techniques later if desired. See - http://seclists.org/nmap-dev/2010/q2/27 [possible SoC student task] - -o Brainstorm for GSoC 2010 ideas and fill out the org application by - Friday 3/12 4PM PST. - o NSE scripts - o Maybe a whole SoC role for http scripts - o Maybe look at other web app scanners for some inspiration - (including w3af - http://w3af.sourceforge.net/) - o Maybe a non-http developer too - o NSE infrastructure manager - o Ncrack - o Nping - o Mobile Devices? N900, iPhone, Android - o Zenmap developer - o Must have solid user interface design experience - o Zenmap script selector (subset of a Zenmap or NSE SoC role) - o Feature Creepers/Bug fixers - -o Review IDS detection scripts from Joao Correa. - http://seclists.org/nmap-dev/2010/q1/814 - -o Review mssql library and scripts from Patrik Karlsson. - http://seclists.org/nmap-dev/2010/q1/1000 (files) - http://seclists.org/nmap-dev/2010/q1/1014 (sample output) - -o Review DNS fuzzer script from Michael Pattrick. - http://seclists.org/nmap-dev/2010/q1/1005 - -o Our nsedoc generator should probably give a warning if a script is - missing any important fields. @output comes to mind. @usage can be - nice too, though we could consider auto-generating that for trivial - scripts. - -o [NSE] Consider pros and cons of splitting information retrieval - scripts into a bunch of small single-purpose script vs. one larger - argument-controlled script. See - http://seclists.org/nmap-dev/2010/q1/1023 - [we ended up combining three of the ms-sql scripts. If we combine - future scripts, we need to remember to add them to the deprecation - list in the Makefile] - -o Remove --interactive. It was broken for a long time and nobody - seemed to notice, and we put a call out on nmap-dev for - --interactive users and didn't get any good reasons to keep it. We - should kill it to remove the code complexity it adds and to avoid - the documentation complexity of people having to read and learn - about a feature they are unlikely to ever use. - -o Zenmanp should perhaps be able to print Nmap output on a Printer (if - not too much of a pain to implement.) - -o Review afp-serverinfo.nse from Andrew Orr. - http://seclists.org/nmap-dev/2010/q1/470 Just waiting on some bug fixes: - http://seclists.org/nmap-dev/2010/q1/665 - -o Test 64-bit pcap installer (e.g. remove old version and install new) - before next release, as we've applied a change from Rob which works on - his system (http://seclists.org/nmap-dev/2010/q1/796). - -o [NSE] Improve username/password library (the database files - themselves). We don't have very good lists at the moment. Maybe - work in combination with Ncrack dev. - o Now there are some even better lists available (f.e. RockYou)--see - this thread: http://seclists.org/nmap-dev/2010/q1/764 - o We've improved the ncrack files--we should probably either use - those for NSE or use a subset of them. - o perhaps from Solar Designer. (he sent us permission) - o perhaps add phpbb hack data (there is at least a list of 28,635 - passwords in phpbb_users.sql, and possibly more in other files. - -o [Nping] Should take the version number 0.[nmap version], such as - 0.5.22TEST - -o Review rpc.lua, nfs-showmount.nse, nfs-get-stats.nse, and - nfs-get-dirlist.nse from Patrik Karlsson. - http://seclists.org/nmap-dev/2010/q1/270 - -o [NSE] Look into moving packet module to C for better performance - [Patrick] - o Removing this one because it is stale (has been here for many - months with no action seen), but it is something we can consider - if/when there is a desire to implement it. A key is probably to - measure current performance and see if it is a material problem. - -o Maybe the Nmap ASCII art should come after make rather than - configure? - - We decided it would probably be annoying for developers to see it - every time they 'make'. - -o Review snmpenum.nse from William Njuguna. - http://seclists.org/nmap-dev/2009/q4/721 - http://seclists.org/nmap-dev/2010/q1/656 - o Dropping for now unless original author or someone else picks it - up and fixes the bugs. - -o Add smtp-enum-users from Duarte Silva if testing is favorable. - http://seclists.org/nmap-dev/2010/q1/699 - -o After the new -sn and -Pn options (added to SVN around 7/20, just - after the 5.00 release) have been around long enough to be in most - people's copy of Nmap (e.g. in all the versions we distribute from - download page (stable+dev)) for at least a few months, we'll document - these as the preferred version rather than -sP and -PN. These match - -n, and the main problem with -sP is that we now use it more for - "disable portscan" than ping only. For example, you can also use - NSE, traceroute, etc. [David] - -o Nmap currently selects routes based on the first matching one it - finds. But it should really take the most specific route instead. - So it should: - 1) Keep searching the routing table for the most specific match, and - 2) Use a stable sort (not qsort) so that routes with identical - netmasks aren't rearranged. - For more, see http://seclists.org/nmap-dev/2010/q1/685 - -o Review pgsql-brute.nse from Patrik Karlsson. - http://seclists.org/nmap-dev/2010/q1/455 - -o psexec missing (need to download yourself now) nmap_services.exe - output issue: "The function where this is detected returns a value - that is passed to stdnse.format_output. format_output takes a - parameter to decide whether it's displaying an error message, but it - is hard-coded to only display error messages with debugging >= 1. So - options are to change format_output and make it more flexible, or - somehow decouple the sensing of nmap_service.exe from the normal - output channel of the script." - -o Website: Create shared directory in svn, which will contain - directories shared between the Insecure.org network of sites - (e.g. templates, error, css). Then sites such as sectools, - nmap.org, insecure.org can just check that out via externals - declaration (or, I suppose, symlink). CSS directives will then use - /shared/css/insecdb.css etc. ). - -o Add CouchDB and JSON scripts once the JSON library is finished. - http://seclists.org/nmap-dev/2010/q1/641 - -o Review NSE raw IP from Kris Katterjohn. - http://seclists.org/nmap-dev/2010/q1/559 - -o Review sslv3-enum.nse from Mak Kolybabi. - http://seclists.org/nmap-dev/2010/q1/563 - -o [NSE] Consider LDAP library and scripts from Patrik Karlsson. - http://seclists.org/nmap-dev/2010/q1/70 [all merged, except David is - still reviewing ldap-search] - -o More potential improvements to http-methods: - http://seclists.org/nmap-dev/2010/q1/630 and - http://seclists.org/nmap-dev/2010/q1/640 - -o Remove smtp-open-relay.nse sometime after 9/24/09 if nobody adopts it (see - http://seclists.org/nmap-dev/2009/q3/0986.html). [It got fixed up - and we kept it.] - -o The -v and -d arguments should take the same syntax. Right now you - use -vvv vs. -d3. We should probably just make either approach work - with either of them. - -o Zenmap should be able to export normal Nmap output - -o Integrate Nping. - -o [NSE] Consider the http-methods script from Bernd Stroessenreuther. - http://seclists.org/nmap-dev/2010/q1/76. [integrated, but David is - making some improvements]. - -o The Nmap web page is beginning to show its age. Ah, who am I - kidding, it was showing its age 5 years ago :). It could do with an - upgrade to XHTML+CSS. It could also do with a whole redesign, but I - think that can be done as a second step after converting to - XHTML+CSS with roughly the same look. Though adding a few more - modern touches (like hover interaction on the menu bar) wouldn't - hurt. This is a moderatly big project, which will involve: o - Designing the new XHTML+CSS to look similar to the current HTML - pages, but be extensible enough that it can be redesigned in the - (near) future by mostly just changing the CSS and graphics. - o Converting the existing Nmap pages to the new XHTML format. - This will likely include using open source programs and likely - modifying them or creating your own scripts to help with the - process. To apply for this task, you need to have some web - development experience and an example XHTML+CSS web page you - have created online. - o We decided not to worry about XHTML for now, and we're - integrating CSS in piece by piece -- we already have the section - headers, left sidebar links. etc. - o Should not use SSI like the current pages -- should do all its - magic through CSS. That way it will work on seclists too (which - can't do SSI for security reasons). - o Maybe alpha transparency for menus, gradiants, curves, etc. But - the main goal isn't flashiness. - -o Seclists.org should maybe be fixed so that it doesn't strip quoted - text for its summaries from the IP list because that list consists - almost entirely of forwarded material which is being stripped. Look - at the summaries at http://seclists.org/interesting-people/. - -o Web site HTML improvements - - Maybe start with nmap.org. - - Find and fix HTML validation problems, bad links. I'm not sure - what tool is best for this. - - Then do the same with seclists.org, insecure.org, sectools.org - - The icon on the top-left of the screen should be for (and link - to) the root URL of current site. e.g. seclists.org, - sectools.org, nmap.org rather than always insecure.org. - -o [NSE] Consider SNMP scripts from Patrik Karlsson. - http://seclists.org/nmap-dev/2010/q1/162 - http://seclists.org/nmap-dev/2010/q1/174 - http://seclists.org/nmap-dev/2010/q1/178 - -o Deal with AV false positive issue RE nmap_services.exe: - - For now, David is going to apply Ron's patch which removes this, - but David will make it print output in verbose mode rather than - debug and maybe make it a little less verbose. LT plan is for Ron - to encrypt it with OpenSSL. - -o Web site improvements - - Update to use CSS, at least for header bars - - Also, if it is easy to give the header bars rounded corners, - we should probably do so. But if it is hard, it isn't - important enough to matter. - - The Nmap.Org navigation table should have a background and more - subtle lines, like we use for our calendars now. - - The first item (table) in featured news has slightly more - left/right margin than the later ones on Firefox 3.5.6, and with - IE8 it doesn't extend as far when you make the page really wide. - Plus the images on the right are problematic (extend through the - border below them) when you make the window too wide on IE8. - Having a slight margin on the left/right of entries would - actually be a bit nice. And it would be nice if it only took a - simple tag or two, controlled by CSS rather than pasting in a - whole table with font tags and the like for each entry. - -o [Ncat] Test, review, and (if appropriate) merge Venkat's HTTP Digest - proxy authentication patch. See - http://seclists.org/nmap-dev/2009/q3/773. [David] - -o [NSE] Look at new DB2 script by Tom - Sellers. http://seclists.org/nmap-dev/2009/q4/659 - -o [NSE] Consider MongoDB scripts and libraries from Martin Holst Swende. - http://seclists.org/nmap-dev/2010/q1/177 - -o [NSE] Document Patrick's worker thread patch in scripting.xml (see - http://seclists.org/nmap-dev/2009/q4/294, - https://nmap.org/nsedoc/lib/stdnse.html#new_thread, - https://nmap.org/nsedoc/lib/nmap.html#condvar) [Patrick] - -o Make Nmap 5.21 bugfix-only release - -o [NSE] Consider afp-showmount script from Patrik Karlsson. - http://seclists.org/nmap-dev/2010/q1/97 - [merged to trunk] - -o [NSE] Review DNS-SD script from Patrik Karlsson. - http://seclists.org/nmap-dev/2010/q1/87 - [merged to trunk] - -o [NSE] Consider MySQL scripts from Patrik Karlsson. - http://seclists.org/nmap-dev/2010/q1/163 - [merged to trunk] - -o [NSE] Consider DAAP script from Patrik Karlsson. - http://seclists.org/nmap-dev/2010/q1/164 - [merged to trunk] - -o NSEDoc left sidebar should include a link to - https://nmap.org/book/nse.html below "Index". - -o Consider enhancing the new OS Assist system to handle version - detection too. [We decided not to do this as David noted that Doug's - serviceunwrap.lisp does pretty much everything he needs.] - -o [NSE] HTTP header parsing is not very robust, and is duplicated in a - lot of places. For example, it's legal to have header fields like -Content-type:\r\n -___text/html\r\n -(with spaces in place of _, but http.lua won't parse such a header -correctly. In other words you can extend them to any number of lines -as long as each line after the first begins with whitespace. [David] - -o Investigate issue with our Pcap and Wireshark x64, as described in - this thread: http://seclists.org/nmap-dev/2009/q4/557 [Rob] - [Taking this off the list until/unless we get more reports] - -o Decide what to do about Windows 7/Vista and starting NPF. See this - thread: http://seclists.org/nmap-dev/2010/q1/20 - -o [NSE] We should do a favicon survey like the one Brandon did for - /favicon.ico files but which uses the favicons specified by the HTML - files rather than just that exact location. For example, insecure.org - sites include in the headers: - - Then we should update our favicon database to include the top ones, - and we should also improve our favicon script so that it either - omits checking /favicon.ico if the HTML-specified one exists, or it - should just download, interpret, and display info for both (right - now it seems to give prority to the wrong one: /favicon.ico). - - -o [Ncat] Add SSL support for --exec so you can use SSL to talk to your - remote shell, etc. See this thread: - http://seclists.org/nmap-dev/2009/q4/255, particularly the - implementation sketch at http://seclists.org/nmap-dev/2009/q4/268 [Venkat,David] - -o Look at new Kerberos script from Patrik Karlsson. - http://seclists.org/nmap-dev/2009/q4/715 . [We decided not to merge - this one since its usefulness turned out to be limited on Windows and - very limited on any other platform. ] - -o Add feature to http library to let user set the user agent to be - used. The NSEDoc for this feature should probably tell what our - current default user agent is ("Mozilla/5.0 (compatible; Nmap - Scripting Engine; https://nmap.org/book/nse.html") [David] - -o On our NSEDoc pages (e.g. https://nmap.org/nsedoc/), perhaps the link - text for scripts should not include the ".nse". Basides saving - horizontal space, this may improve the sorting so that the likes of - "citrix-enum-apps" comes before "citrix-enum-apps-xml". Also, we can - probably get away with reducing the width of the NSEDoc left-column, - especially if ".nse" is removed. - -o [NSE] Patrick's script dependency patch: - http://seclists.org/nmap-dev/2009/q4/295 - o I'm not sure if he has gone through and actually set appropriate - dependencies (and removed runlevels) yet - -o Integrate latest version detection submissions and corrections. - This was last done based on submissions until February 9, 2009. - -o Release 5.10BETA2 - -o Add --evil to set the RFC3514 evil bit. - ftp://ftp.rfc-editor.org/in-notes/rfc3514.txt - o We're not going to add this right now. - -o Talk to Libpcap folks about incorporating (at least some of) my - changes from libpcap/NMAP_MODIFICATIONS. [marking as done since the - upstream-appropriate changes are pretty minor now that we've - upgraded to 1.0] - -o Nping -- like hping3 but uses Nmap infrastructure and to a - large degree the same command-line options as Nmap. - [We now have an alpha version at https://nmap.org/nping/] - -o Further investigate SCTP functionality, as some people reported - problems (see this thread: - http://seclists.org/nmap-dev/2009/q2/0669.html) - -o [NSE] NFS query script for checking exports, etc.? [Patrik Karlsson] - -o [NSE] Attempt to reproduce and fix a deadlock reported by Brandon - when he does large-scale scanning with a new favicon script with - hostgroups as small as 8,192 (he hasn't seen it with 4096 - hostgroups). Could be a bug in internal NSE socket lock. Probably - not specific to the favicon script, but that is how Brandon - reproduces it. At the hang, stack trace is usually the threads stuck - in socket_lock function, sometimes lookup_cache mutex in http - library. David guesses that it's threads being garbage-collected - from the socket lock table. The only thing that can wake up a thread - waiting on a socket lock is if a thread that holds a lock is removed - from the table. But the table has weak keys, meaning that a thread - can be garbage collected and it will be automatically removed from - the table by the Lua runtime. Then there is no event that can wake - up a thread waiting for a lock. [David and Patrick made some commits - at end of November meant to resolve this, and we haven't seen the - problem since, so we're marking it as done for now]. - -o Look into reducing Nmap memory consumption - o UDP scans with -p- and large hostgroups are a particularly large - offender. See if there is a way to prevent them from eating up - gigs of RAM. See the message "Port memory bloat" at - http://seclists.org/nmap-dev/2009/q3/0926.html for a patch that - reduces Port memory use by about 50%. - o One idea David has been considering is a way to represent filtered - ports (or whatever the default state is) without creating a Port - object for each one. - [David] - -o Fix assertion failure with certain --exclude arguments (see - http://seclists.org/nmap-dev/2009/q4/276). [David] - -o Many people may have stale (since removed/renamed) scripts in their - Nmap scripts directory because our 'make install' does not remove - them and so they remain and can cause problems (like running twice - after being renamed). We should probably add a line to our 'make - install' which removes the scripts/lib names we have previously - used. We're doing this rather than blowing away the old directory - just in case someone has custom scripts/libs there (though that is - still a bad idea). [David] - -o Update the CHANGELOG for new 5.10BETA1 - release. [Fyodor] - -o Make the new Nmap 5.10BETA1 release - -o Ndiff man page should be built from XML source whenever a release is - done, as ncat/zenmap/nmap man pages are. [Fyodor] - -o We should package the rendered Nroff man page translations (e.g. all - 16 languages) in the tarball to make it easier for distributors to - package them. For example, see - http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=358336. Including - the translations would add 2.5MB to the (currently 28MB) - uncompressed tarball and about 800KB to the (currently 9MB) bz2 - compressed tarball. [Fyodor] - -o The Nmap 5.00 tarball contains: - -rw-r--r-- fyodor/fyodor 122943 2009-06-24 14:35 nmap-5.00/docs/scripting.xml - -rw-r--r-- fyodor/fyodor 151 2009-06-24 14:35 nmap-5.00/docs/nmap-usage.xml - -rw-r--r-- fyodor/fyodor 604 2009-06-24 14:35 nmap-5.00/docs/nmap-man-enclosure.xml - -rw-r--r-- fyodor/fyodor 76918 2009-06-24 14:35 nmap-5.00/docs/nmap-install.xml - -rw-r--r-- fyodor/fyodor 10179 2009-06-24 14:35 nmap-5.00/docs/legal-notices.xml - If we're going to include the XML source files, we should include - refguide too. But rather than add that, we should probably take - these out. After all, people can easily grab them from svn or our - new http svn gateway if desired. So no need to bloat the tarball - with these files which aren't installed. [We're going to take the - XML source files out of the tarball] [Fyodor] - -o Consider converting this file to emacs org-mode - (http://orgmode.org/) format. [Fyodor] - o That format is still plain text and can be read/edited by vi - users, etc. - [Considered, but I don't think I'll change right now] - -o Windows 7 RTM Nmap testing (With particular attention to 64-bit and - our pcap installer). [Fyodor] - -o We should print host latency (when available) in the XML output, as - suggested at http://seclists.org/nmap-dev/2009/q4/215. - docs/nmap.dtd will have to be modified accordingly, and you might - even consider adding support to docs/nmap.xsl. - -o Integrate latest OS fingerprint submissions and corrections. This - was last done based on submissions up to May 8, 2009. - -o Potential OS X 10.6 problems. There are two issues reported by the - same user which may be related: - http://seclists.org/nmap-dev/2009/q3/0936.html, - http://seclists.org/nmap-dev/2009/q3/0996.html. One is that Nmap - hangs doing nothing and needs to be killed with Ctrl-C, and the - other is that it dies after printing "Initiating UDP Scan". Another - reported the same problem at - http://seclists.org/nmap-dev/2009/q3/0990.html, where it dies after - the first ARP request is sent. But Brandon has run Nmap on 10.6 - without problems. It is a bit of a mystery. [David] [Resolution: - Apple fixed the problems in 10.6.2; For users who have 10.6 and - 10.6.1, the versions David builds on 10.5 will still work for them - because they are 32-bit binaries rather than 64. Users who build - Nmap on 10.6 or 10.6.1 should compile with -m32 or update to 10.6.2] - -o [NSE] Patrick's worker thread patch: - http://seclists.org/nmap-dev/2009/q4/294 - -o Investigate get_rpc_results error (infinite loop) reported by Lionel - Cons. See these threads: http://seclists.org/nmap-dev/2009/q4/24, - http://seclists.org/nmap-dev/2009/q4/120 - -o Upgrade to latest version of NSIS on Nmap Win build system [Fyodor]. - -o Standardize on a proper file header for the Zenmap source code. [David] - o For now, David is going to augment the templatereplacement system - to insert the normal nmap.header.tmpl, but change the comment format - to work with Python, and then replace the current Zenmap headers - with that. - -o We may want to look into if/how we support IPv6 nameservers. Here - is a bug report from someone having a problem with them: - http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=539244 [Ankur] - -o Once all the man page languages are in the Nmap tarball, we should - update our install system to install them in the appropriate place. - We'll want to integrate this with configure so users can decide which - languages they want. See http://seclists.org/nmap-dev/2009/q4/249. - -o Resolve allow_ipid_match issue which can cause some malformed - replies to be ignored when we might be able to still use them. See - this thread: http://seclists.org/nmap-dev/2009/q2/665 [David] - -o Fix Zenmap 'make install' TypeError issue - (http://seclists.org/nmap-dev/2009/q4/225). [David] - -o Fix a bug in which Nmap can wrongly associate responses to SYN and - ACK host discovery probes. [David] - For example: - # nmap -sP -PS80 -PA80 australia.gov.au --packet-trace -d2 - SENT (0.0760s) TCP 192.168.0.21:60182 > 152.91.126.70:80 S ttl=43 id=13466 iplen=44 seq=4046449223 win=4096 - SENT (0.0770s) TCP 192.168.0.21:60182 > 152.91.126.70:80 A ttl=48 id=39976 iplen=40 seq=4046449223 win=1024 ack=921915001 - RCVD (0.3020s) TCP 152.91.126.70:80 > 192.168.0.21:60182 SA ttl=53 id=0 iplen=44 seq=3924706636 win=5840 ack=4046449224 - We got a TCP ping packet back from 152.91.126.70 port 80 (trynum = 0) - ultrascan_host_probe_update called for machine 152.91.126.70 state UNKNOWN -> HOST_UP (trynum 0 time: 226875) Changing ping technique for 152.91.126.70 to tcp to port 80; flags: A - In the example above, Nmap wrongly uses ACK as the preferred ping technique, when it should be SYN. [David] - o we're thinking about ways to encode the information better. Right - now we have pingseq and tryno, but we may want to just move to a - single probe ID and then we can look up any other information in - structures attached to that ID in memory when we get the response. - o A related problem, which we hope the fix for this will also - resolve, is that replies can currently match any probe whose tryno - is less than or equal to the tryno encoded in the reply. - o However, "fixing" this problem has been shown in the past to - cause accuracy problems. See - http://seclists.org/nmap-dev/2009/q1/387. We should figure out - whether we can still reproduce that and, if so, what is going on - before "fixing" this issue. - -o Add PJL (Printer Job Language) probes to - nmap-service-probes. Brandon wrote some in - http://seclists.org/nmap-dev/2009/q1/0560.html. Test them to see if - they cause anything to be printed out (on paper) with printers that - don't support PJL. If not, then remove the JetDirect ports from the - default exclude list. The script pjl-ready-message.nse also uses - PJL. We have concerns about the safety of this probe given - http://seclists.org/nmap-dev/2009/q4/61, but it still is probably - better to have the probe in there than not, as long as we continue - blocking the ports by default with the Exclude directive. - [We put in the probes, but are keeping the Exclude directives - because the probes still seem a bit dangerous] - -o [NSE] in_chksum in packet.lua doesn't work with an odd number of - bytes. Also make it more efficient. - -o Add --confdir option to Zenmap. See - http://seclists.org/nmap-dev/2009/q1/92 [David] - -o Update our Winpcap from 4.0.2 to 4.1.1 - (http://seclists.org/nmap-dev/2009/q4/128). This is a bit complex - because we have our own installer. See - https://nmap.org/svn/mswin32/winpcap/Upgrading-Instructions.txt. - -o Change Nmap to not show the "Host not scanned" lines in list scan - -o Change Nmap to show latency in "host is up" lines even w/o verbose - mode. - -o Update our included Libpcap from 0.9.7 to 1.0.0 - (http://www.tcpdump.org/) [David] - -o Improve Nmap output to show the forward DNS name when specified on - command line as well as rDNS where appropriate. We're also going to - reorganize output to enable some other improvements as well. See - the proposal at http://seclists.org/nmap-dev/2009/q3/814, and that - whole thread which starts at - http://seclists.org/nmap-dev/2009/q3/805 [David]. - -o [Zenmap] Solve some unusual utf8 Zenmap crashes reported in the - crash reporter. David has fixed some of them so far, but there are a - few more remaining that may be related. [David] - -o Change Nsock to give an error if you try to FD_SET a fd larger than - FD_SETSIZE. [Brandon] - o Some research from David: - We have help off on this change because of Windows portability - problems. The Windows fd_set works differently than the Unix - fd_set. In Unix, FD_SETSIZE (which is typically 1024) is both the - maximum number of file descriptors that can be in the set and one - greater than the greatest file descriptor number that can be - set. In other words, we want to bail out whenever someone tries - to FD_SET file descriptor 1060, for example. But on Windows it's - different: FD_SETSIZE is only 64, but any file descriptor - numbers, no matter how great, may be stored in the set. Windows - socket descriptors are typically greater than 1023, but you can - only have 64 of them in the set at once. - - So the fix on Unix would be - --- nsock/src/nsock_core.c (revision 15214) - +++ nsock/src/nsock_core.c (working copy) - @@ -97,6 +97,7 @@ - do { \ - assert((count) >= 0); \ - (count)++; \ - + assert((sd) < FD_SETSIZE); \ - FD_SET((sd), (fdset)); \ - (max_sd) = MAX((max_sd), (sd)); \ - return 1; \ - @@ -107,6 +108,7 @@ - assert((count) > 0); \ - (count)--; \ - if ((count) == 0) { \ - + assert((sd) < FD_SETSIZE); \ - FD_CLR((sd), (fdset)); \ - assert((iod)->events_pending > 0); \ - if ((iod)->events_pending == 1 && (max_sd) == (sd)) \ - - But that doesn't work on Windows (I just tried it) because even - the smallest socket descriptor is bigger than FD_SETSIZE, 64. - Really we're trying to accomplish two different things on the two - platforms: On Unix we must not store a file descriptor greater - than 1023, no matter how many or how few other descriptors have - been set. On Windows we must not set more than 64 descriptors at - a time, no matter what their descriptor number happens to be. - -o Add a way in NSE to set socket source addresses and port numbers. - See this thread: http://seclists.org/nmap-dev/2009/q3/821. Some - potential solutions are discussed later in the thread. - -o [Ncat] Fix --max-conns on Windows so that it only counts concurrent - connections and not long-dead ones. See this thread - (http://seclists.org/nmap-dev/2009/q3/1017.html) and particularly this - message (http://seclists.org/nmap-dev/2009/q3/1032.html) for - details. Venkat has a patch for David to review and potentially merge. - -o [Ncat] Fix 100% CPU usage with ncat -l --send-only. See this - thread: http://seclists.org/nmap-dev/2009/q2/797 and continues - further at http://seclists.org/nmap-dev/2009/q3/99. This message is - key: http://seclists.org/nmap-dev/2009/q3/308 [David] - -o [Seclists] There is currently some extra vertical space after the - first post of a thread in the thread index (example: - http://seclists.org/nmap-dev/2009/q4/index.html). - -o [NSE] Decide which scripts belong to the "safe" category (we now have 20 - which aren't either safe or intrusive), then remove the intrusive - category since people can now specify "not safe". See - http://seclists.org/nmap-dev/2009/q3/1091.html and that whole - thread. [Fyodor] - [ OK, see http://seclists.org/nmap-dev/2009/q4/0002.html] - -o [NSE] Fix http pipelining. Responses are being split on anything - that looks like HTTP/1.X which doesn't come at the beginning of a - line, and doesn't work when a line like that happens to legitimately - come in a body. Joao has an nmap-exp branch which resolves this - issue, though David found some bugs in that and sent some hard test - cases. [Joao] - -o Fix traceroute performance/algorithms. It is terribly bad in some - cases. For example, this traceroute scan took 36 minutes against a - single host(!): http://seclists.org/nmap-dev/2009/q3/0425.html . We - don't need to go up to hop 50 in such cases (maybe some heuristic - like "at least go to hop 15, and stop after 5 unresolved in a row). - And more importantly, there is no reason each hop should take 40s to - timeout. It should probably use timeout variables like we use in - port scanning. And it should parallelize as much as possible. Even - if parallel resolution means we went a little further than we had to - in incrementing the TTL, and we go to hop 15 when host is at 12 - that's no big deal (of course we would only report up to hop 12 in - the output). Once we do this, we should put back the ability to - make --traceroute work even when we haven't found a probe which - elicits a response from the target. (that feature was added in July, - but we'll probably take it out until we can fix - performance). [David] - -o Fix four Nmap bugs discovered by Ankur and analyzed a bit by - David. [Ankur] - -o [NSE] Consider HTTP request caching. - -o [NSE] Finish (or write new) favicon fingerprinting script. See - http://seclists.org/nmap-dev/2008/q4/0583.html . May need to do - some more scanning and increase the DB size a bit. May or may not - want to later combine this as part of a larger webapp fingerprinting - script. - -o [Zenmap] When the inventory is changed, the current host/service selection is - forgotten and the Ports / Hosts tab is switched to hosts mode. It should - remember your current selection and not change the view. [David/SoC] - -o Device categorization improvements - o Examine Nmap's device categorization in nmap-os-deb and - nmap-service-probes. Decide if some small categories which have - never really took off should be consolidated, or whether others - should be split off. For example, maybe there are some groups in - 'specialized' or other misc. categories which are now large enough - to split off. Personally, I wouldn't give anything its own - category unless there are at least half a dozen of them and no - other category really fits them well. We should use a combined - system for nmap-os-db and nmap-service-probes. - o Add a classification sect1 to os-detection.xml - (https://nmap.org/book/osdetect.html) to cover how Nmap handles OS - classification. It should include a list with descriptions of - each device type recognized by Nmap. Version-detection.xml should - reference (link to) it in the approprate place. - [Doug has done some initial work on this. For example, see - nmap/docs/device-types.txt] [David] - -o Consider what new UDP payloads we might want to add. David has many - ideas at: http://seclists.org/nmap-dev/2009/q3/0290.html - -o For traceroute we should give some indication that the RTT is in ms. - Changing the column header to maybe "RTT MS" or "RTT (MS)" would - probably do the trick or we could append "ms" to each value. - [David] - -o OS fingerprint should probably specify somewhow when DS=1 if it's - because target->directlyConnected is true, or because it sent the - distance probe and calculated a distance of 1. The second situation - should never happen, but often David strongly suspects that it is the - case. - -o --traceroute should probably set currenths->distance because right - now, I do an -O scan against scanme.nmap.org, and it does not figure - out the distance. So the fingerprint shows no distance element and - Nmap doesn't print "Network Distance" in the results line. That may - be OK (Nmap probably isn't receiving the probe response needed for - this, and maybe doesn't want to print the TG), but even when I do - --traceroute I get no distance printed. Yet Nmap clearly knows the - distance since the traceroute shows all the hops up to and including - the target (scanme.nmap.org). - -o Figure out best favicon to use for Nmap and related web sites - [David] - -o [Ncat] David says: "After you get EOF on stdin with --send-only, the - program hangs on until the idle timeout expires instead of terminating - immediately. I had a fix for it but it involved deleting events in - the Nsock queue and it caused an assertion failure in Nmap so I backed - it out. I have a less intrusive solution." [David] - -o We should update our config.{sub,guess} files. This Debian bug - #542079 requests that we do so: - http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=542079. We last - updated on 3/15/08 and in that case we used versions from - http://cvs.savannah.gnu.org/viewvc/config/?root=config. That may or - may not be the best place to get them now (e.g. perhaps there has - been a recent official release). [David] - -o Look a bit more at default version detection timing. Particularly - deciding the number of probes to run in parallel. [ We increased - that a bit on 8/18/09] - -o [Ncat] Right now our -i (idle timeout) causes Ncat to quit if EITHER - reading or writing is idle for the given amount of time. But it is - really only idle if BOTH reading AND writing are idle for the - period. We should make the code work that way. - -o Add scripting.xml documentation on strict.lua and the avoidance of - global vars in libraries. See - http://seclists.org/nmap-dev/2009/q3/0169.html. Probably a new - section just above "Adding C Modules to "Nselib", such as "Writing - Your Own Library" or somesuch. [Patrick] - -o Update nsedoc to refer to 'libraries' rather than 'modules'. This - affects the front page (which calls them 'Libraries' on left sidebar - and 'Modules' on the list of right, and affects the url (we should - change /modules/ to /lib/ and then have Fyodor add a redirect for - people still using old URLs) and the title of the module pages like - https://nmap.org/nsedoc/modules/base64.html. [Patrick] - -o [Ncat] Prefix Ncat stderr messages with "Ncat: " to make it clear - that they are coming from Ncat and not the remote server (or typed in - by user). [David/SoC] - -o [NSE] Optimize NSE Performance--e.g. measure the current performance and - see what can be improved in terms of scheduling scan threads, - determining how many to run concurrently, looking at CPU load items, - etc. [David/Patrick] - -o Increase version scan concurrency based on Patrick's performance - testing. We decided to go to 20 for timing_level 3, 30 for 4, and 50 - for 5. - -o [NSE] Consider POST/HEAD support. See - http://seclists.org/nmap-dev/2009/q1/0889.html. - o Implemented: http://seclists.org/nmap-dev/2009/q3/0074.html - o Joao going to check in very soon soon. - -o [NSE] Consider Rob Nicholls http-enum script for incorporation: - http://seclists.org/nmap-dev/2009/q1/0889.html - [Joao tested w/his HEAD support, is going to check this in] - -o Consider the open proxy scripts more carefully - - How should we test whether the proxy attempt was successful? Right - now we look for a google-specific Server header after trying to - reach http://www.google.com through the proxy. Maybe we should let - users specify their own pattern if they specify their own URL. - [ Joao is going to check it in today (7/28)] - -o I should add code to Nmap to bail if sizeof(char) isn't 1. - Otherwise there could be security risks if it is not one on any - platforms. [ Actually, we think C standard requires this and we've - not heard of any system where sizeof(char) isn't 1. So removing - this item.] - -o [Zenmap] More complete implementation of ZenmapCommandLine/profile - editor improvement ideas. See - http://www.bamsoftware.com/wiki/Nmap/ZenmapCommandLine. [David] - -o [Ncat] Think about whether we should offer "-q secs" (quit after EOF - + delay of secs) and/or -k (set SO_KEEPALIVE on socket) (or maybe - that should be set by default). Anyway, these were suggested here: - http://lwn.net/Articles/341706/ [We're going to fix -i (added - separate item), and not worry about SO_KEEPALIVE unless we see more - demand for it. It doesn't seem that nc110 or OpenBSD nc or so-called - GNU Netcat support SO_KEEPALIVE either] - -o [Ncat] In verbose mode, I'd like to see clock time (duration) and - maybe in/out traffic stats when a client connection ends. Maybe it - could use a format similar to what Nmap provides. [David/Venkat] - -o Seriously consider making --traceroute work even when we haven't - found a probe which elicits a response from the target. We'd just - have to pick a probe in that case (probably echo request, as we - found that to be the most effective in prev. empirical testing). - This is similar to UNIX traceroute and Windows tracert.exe which - just pick a probe (high UDP port on UNIX, ICMP echo request on Win). - Even if the host is down or something, we usually get some useful - hop information. - -o [NSE] Allow spaces in script arguments without the user having to - manually quote them (beyond normal shell escape quoting). See: - http://seclists.org/nmap-dev/2009/q3/0090.html - [Patrick] - -o [Ncat] Support SCTP now that Nmap does. - - See client support patch by Daniel Roethlisberger: - http://seclists.org/nmap-dev/2009/q2/0609.html - - Server support? - - Daniel has a patch, David looking to apply once an nsock thing is fixed. - -o Look at etc/payloads.conf in unicornscan-0.4.7 and see if they have - any which we don't have, but should, for our version detection. - They have a decent collection there. KX sent some other programs we - should look at too. [David] - -o Ncat should give it's ethernet cat ASCII logo after - configure--similar to the way that Nmap, Ncrack, and Nping - do. [David/SoC] - -o [Zenmap] The Search dialogue is helpful for finding a certain scan - you've performed recently, but we should probably also offer a similar - function for searching for certain applications/hosts within a scan - (e.g. find all the hosts running Apache). This new functionality - might be a find option or some other mechanism rather than being - part of the Search dialogue proper. - -o Ncat SSLv2 issues. See - http://seclists.org/nmap-dev/2009/q1/0319.html. A big part of it is - done, which was enhanced version detection probes to detect more SSL - servers, The defect that remains is that Nsock can't connect to a - small fraction of servers (including some of the ones detected by - the new version probe). They are the servers that do only SSLv3 or - TLSv1 and don't respond to a SSLv2-compatible ClientHello. Even - though most servers don't support SSLv2, they usually respond to the - ClientHello and just don't offer any SSLv2 features. [David/Venkat - working on this] - -o Deadlock identification and correction: - o Plan of action: implement freeing of script mutexes when scripts - exit without freeing them (done and in /nmap now). And then if it - continues to be a problem we'll consider this other stuff: - o Add detection for deadlocks and print which threads are involved. - o use above results to make a strategy for automatic deadlock resolution. - o Original entry: Figure out what to do about NSE mutexes: - http://seclists.org/nmap-dev/2008/q3/0276.html . In particular, they - are not currently cleaned up if a thread dies or otherwise exits - without unlocking them and can cause endless deadlocks which are - annoying to users and can be difficult to debug :(. Patrick has - some ideas for this in his SoC09 proposal: - "Adding a cleanup system for NSE that is called periodically - similar to nsock_loop. There would be a registration system - allowing C libraries to register a Lua function that will run - periodically to check for irresolvable deadlock or simply dead - resources. For example, the nmap library would register a mutex - cleanup handler which would inspect all mutexes looking for a dead - thread or circular dependencies. The nsock library could register - a handler that checks for unused sockets. The nsock may save a - strong reference to the thread that owns the socket and inspect it - to determine if the thread is dead." - David later says: "After some discussion we decided to start more - modestly, first by ensuring that a scripts mutexes are released when - it dies for whatever reason. I have a hunch that this is the cause - of most deadlocks. It was certainly the cause of two whois.nse - deadlocks I found. Then, the next step if deadlocks continue to be a - problem, is to do automatic detection and just print out a list of - what scripts are involved. It could be that several smb scripts are - deadlocked, or as in the case I observed where whois.nse was locked - with itself." - -o Joao is auditing his Lua code to make sure all his variables are - local where appropriate. [Joao - done, should be commited very soon] - -o [NSE] We need to deal with libraries which improperly use global - variables, as that is very common (Patrick made a list: - http://batbytes.com/bad.txt). Solutions could involve augmenting - our runtime system (the "strict.lua" approach) to detect/prevent the - problem, a script we run occasionally to identify issues that we - then manually resolve, or, at the very minimum, documenting - somewhere in scripting.xml the dangers inherent in global variables - and warn people to generally declare them local instead. We have a - long history of bugs caused by non-local variables defined in NSE - libraies and often causing deadlocks. - -o The Nmap refguide (https://nmap.org/book/man-performance.html) says - "The --max-parallelism option is sometimes set to one to prevent Nmap - from sending more than one probe at a time to hosts. This can be - useful in combination with --scan-delay (discussed later), although - the latter usually serves the purpose well enough by itself." But - when you actually try it: - # ./nmap --max-parallelism 1 --scan-delay 10 scanme.nmap.org - You can't use --max-parallelism with --scan-delay. - QUITTING! - We need to either make that work or adjust the documentation. [David/SoC] - o David changed this to a warning. Note that with --scan-dealy, - --max-parallelism is essentially 1 anyway. - -o [NSE] Consider integrating HP Laserjet print PJL status-setting - script. See this thread for an example of such a script: - http://seclists.org/nmap-dev/2009/q3/0083.html (note that it is - updated during the thread). Also, see this thread: - http://seclists.org/nmap-dev/2009/q3/0092.html - -o Ndiff man page should be expanded to include sample execution/output - and more fully describe its functionality. [David] - -o David is going to reexamine the old coverity-reported issues (the - ones we previously marked as "ignore" because they weren't real bugs) - just to be sure that is (and is still) the case. - -o Make -sP work with -PN to disable both port and ping scanning. We - need to make sure the various options still work (-O, --script, - --traceroute, etc.) with this, as many currently don't as they don't - expect this behavior, which used to be unsupported and cause Nmap to - quit with an error messaqge. It may be OK to refuse -O since that - will rarely give useful results. OTOH, -O may work on some systems - with unique closed port signatures where Nmap guesses a closed - port. Users should then be able to do an NSE-only scan with "-sP -PN - --script [scripts]" We should document this -sP -PN usage in - refguide. [David] - -o Add -sn and -Pn options which are aliases for -sP and -PN. Once - they've been around long enough to be in most people's copy of Nmap, - we plan to document those as the preferred version. Those match -n, - and the main problem with -sP is that we now use it more for - "disable portscan" than ping only. For example, you still might - want to use NSE. [David] - -o [NSE] Make sure all our HTTP scripts transparently support SSL - servers too. [Joao has a solution and is testing the http scripts to - make sure they don't break.] - -o Resolve "memcpy overlap in getinterfaces(int*) (tcpip.cc:2987)". - See this thread: http://seclists.org/nmap-dev/2009/q2/0713.html - [David/Brandon] - -o [Ncat] Print a message to stderr upon connection failure even if -v - isn't specified so the user knows what went wrong. [David/SoC] - -o [Ncat] Maybe --chat should imply -l. And Maybe --broker should too? - - OTOH, we might want to extend --chat for connect mode in the - future. - [We're going to hold off on chat now, David/SoC is doing --broker] - -o Consider making it easier to tell whether scripts were specified by - name on the command-line (rather than default or by class) so they - have the option of providing extra verbosity in that case. For - example, see http://seclists.org/nmap-dev/2009/q2/0563.html. We - could either provide a special function for scripts to determine - that, or we could magically adjust nmap.verbosity() when called by - those scripts. [David] - -o [NSE] Figure out a way to support people who want to do script scan, - but not port scan or ping scan. One option would be to allow - --script to list scan (-sL), but perhaps a better option is to - provide a way to disable port scanning in the same way as we offer - -PN to disable ping scanning. As an example of this need, David had - to write special code to avoid ping/port scanning when doing a - whois.nse survey for - http://www.bamsoftware.com/wiki/Nmap/EffectivenessOfPingProbes. The - key for this task is to figure out how to do it from a user - interface perspective and then implement and document it. We've - already been going in the direction of allowing script scanning in - more types of scans--a while back we started allowing it with -sP - ping scans due to high demand. [David/SoC] - [ We decided how we're going to do it (-sP -PN to start out with; - leading to eventual -sn -Pn) and added new TODO entries for actually - doing the code/docs. ] - -o Ndiff should be able to show NSE script result changes. [David] - -o Get set up for Coverity scan of latest version to see if it catches - any important issues before stable release. [Fyodor,David] - [Found 7 new results, 3 are real bugs, and 2 have been fixed so far] - -o [nsock] Fix Makefile to handle dependencies correctly (if that turns - out to be the problem). See - http://seclists.org/nmap-dev/2009/q1/0629.html. o Or it may be - related to SVN timestampling. See - http://seclists.org/nmap-dev/2009/q1/0632.html. Diagnosed by David: - http://seclists.org/nmap-dev/2009/q2/0728.html - -o For at least our UDP ping probes, Nmap should probably notice if it - is a very well known service port such as 53, 161, or 137 and send - an appropriate probe packet (server status for DNS, public community - string query for SNMP, etc) rather than empty data in that case. - This is similar to the way our IP protocol probes automatically - include common headers such as TCP and UDP if that common protocol - is given. Good probes for these services are already available in - nmap-service-probes, though we might want to make a custom file for - this. We should probably do this for port scanning as well. [David] - -o [NSE] Make NSE work better for SSL tunneled services in general by - supporting them easily in the libraries. For example, I don't think - irc-info.nse currently works against all the servers which tunnel - over SSL. Maybe augment comm library, etc. [Joao - done, except for - http, which is already a separate TODO item] - -o Update scripts which use table args to use pseudo-table format - "name.arg" rather than requiring the user to create a Lua table - themselves. On the lua side, it's not really being stored in a - table, but just an arg named "name.arg". [Joao] - - Look at all our existing scripts which use tables - (dns-zone-transfer, whois, the proxy scripts, etc.) and change as - appropriate. Remember to change the usage throughout the script - and also change the nsedoc script arguments and example usage. - For the existing scripts, try to retain the table version check - for now to avoid breaing backward compatability if possible. Just - add the newer style check as well. - - Is taking arguments in a table specific to a script a good idea? - The example in the socks-open-proxy nsedoc of "--script-args - openproxy={host=}" is a bit of a mess and I'm not sure the - best way to document that in the script argument list. Note that - this is the standard way we've handled it for some other scripts, - so it's not an open-proxy-script-specific problem. - -o [NSE] Track active sockets in the nsock library binding and don't - rely on garbage collection for reallocation. Can probably wait until - post-stable release for integration. [Patrick] - - Patrick has a patch and is waiting on dev branch to check it in. - -o [NSE] Resolve ssh2.lua buffering problems - (http://seclists.org/nmap-dev/2009/q2/0673.html) [Joao] - -o Decide what to do about ncat source code headers -- maybe just use - the Nmap ones. [David added the Nmap headers] - -o Once we go into deep stability freeze mode, create an nmap-exp - development branches for changes we plan to integrate after the - stable release. [Fyodor] - -o Update CHANGELOG for latest changes [Fyodor] - -o Release 4.85BETA10 - -o [NSE] Open proxy detection scripts - o We have http-open-proxy.nse, but we should probably either extrand - that to handle other types of proxies (such as SOCKS and HTTP - CONNECT) or create more scripts to handle those other proxy - types. [Joao, David] - o Joao has written scripts, just need to finish up, evaluate, integrate. - -o Determine whether zenmap.spec.in can currently require - "python-sqlite" rather than "python-sqlite2", or if it at least can - be easily made to do so. The former seems more compatible since - RHEL/CentOS 5.3 has a "python-sqlite" package, but not - "python-sqlite2". Meanwhile, Fedora 10 provides the "python-sqlite" - capability as long as you have the Python 2.5 package installed - (python-2.5.2-1.fc10). Fedora 10 does also make a - python-sqlite2 package available. - -o [Ncat] Solve EOF issues which crop up when piping to an external - command. See http://seclists.org/nmap-dev/2009/q2/0528.html. It - sounds like we will go with Daniel's patch [Daniel, David] - -o Look into building RPMs with SSL support. Statically linking to - OpenSSL on Linux for the RPMs didn't work for me last time I - tried. [Fyodor] - o Static linking of Nmap to OpenSSL does not seem to work on Fedora - 10 or CentOS 5.3. The problem appears to relate to the OpenSSL - krb5 support. - o Could build my own OpenSSL libraries on the build system - (w/o Kerberos support) and link to those. - o At some point, we might want to consider including OpenSSL with - Nmap tarball. The problem is that it is rather big. Would - increase Nmap .tar.bz2 size from about 9 megs to about 12. OTOH, - OpenSSL is only going to get more and more important. Maybe we - can include a stripped down version? - o If we don't integrate OpenSSL (or until we do), we might consider - a more prominent configure warning for when SSL is not detected. - We could suggest that users run "yum install libopenssl-devel" or - "apt-get install libssl-dev" commands or whatever is appropriate - and then reconfigure. Or we could point them to a page or - nmap-dev posting URL with instructions. - -o Figure out why I [Fyodor] get a bunch of "Operation not permitted" errors -when I launch a scan on SYN such as: - - I'm going to ignore this for now unless it causes me trouble - again, as this is an old machine that will be replaced soon anyway. - And we haven't been hearing of the problems from others lately. - /home/fyodor/nmap-exp/fyodor-perf/nmap -nogcc -T4 -n -v -p- --portpingfreq 250 -oA /home/fyodor/nmap-misc/logs/WorldScan/portpingfreq/logs/portpingfreq-250-1%T-%D 67.15.236.34 67.15.236.36 81.174.236.66 81.174.236.119 170.140.20.160 170.140.20.174 202.138.180.9 202.138.180.17 202.138.180.132 209.20.64.112 - The errors look like: -sendto in send_ip_packet: sendto(7, packet, 44, 0, 170.140.20.174, 16) => Operation not permitted -Offending packet: TCP 64.13.134.4:59820 > 170.140.20.174:59120 S ttl=39 id=19927 iplen=44 seq=2425535549 win=4096 -sendto in send_ip_packet: sendto(7, packet, 44, 0, 67.15.236.36, 16) => Operation not permitted -Offending packet: TCP 64.13.134.4:59820 > 67.15.236.36:15030 S ttl=57 id=50640 iplen=44 seq=2425535549 win=2048 -Discovered open port 49394/tcp on 170.140.20.174 -sendto in send_ip_packet: sendto(7, packet, 44, 0, 170.140.20.174, 16) => Operation not permitted -Offending packet: TCP 64.13.134.4:59819 > 170.140.20.174:8256 S ttl=48 id=38510 iplen=44 seq=2425601084 win=1024 - May be related to connection tracking and high scan rates. See - http://seclists.org/nmap-dev/2008/q4/0652.html - http://www.shorewall.net/FAQ.htm#faq26 - Others have reported similar issues even without connection tracking. See - http://seclists.org/nmap-dev/2006/q3/0277.html - http://seclists.org/nmap-dev/2007/q2/0292.html - - -o -PO1 and "-sO -p1" seem to send ICMP ping packets with an ICMP ID - field of 0, which we found that a small percentage of hosts drop - (61.13% responded with 0, 62% with a random value). So we might as - well randomize them in these cases. [Josh Marlow] - -o Some of the -PS443 scans (and maybe other ones) we've been running - have been missing the Nmap line telling how many packets were - sent/received, even though we had verbose mode. [David/Josh] - -o Deal with Ncat newline problem. See this thread: - http://seclists.org/nmap-dev/2009/q2/0325.html [David,Jah] - -o Integrate SCTP scanning support. See Daniel Roethlisberger's branch - in nmap-exp/daniel/nmap-sctp. As of 4/30/09, he is nearing - completion. See http://seclists.org/nmap-dev/2009/q2/0270.html. - -o [NSE] Release mutexes upon script death to prevent certain deadlocks - [Patrick, David] - -o Consider whether to let Zenmap Topology graph export the images to - svg/png/etc. Also think about printing. Note that João Medeiros - has written a Umit patch to do this: [Joao, David] - http://trac.umitproject.org/ticket/316. - - Now he has Nmap patch: - http://seclists.org/nmap-dev/2009/q2/0409.html - - Consider integrating. - - Integrated! - -o Ensure that when I build a distribution package on UNIX (e.g. make - distro), it builds what is in the Nmap directory I am calling it - from rather than a particular SVN version. I'm going to start - building packages from a special "clean" directory which is - different than the one I do development work in. Also, I want to be - sure that any changes in that dir are included in the release, even - if they aren't check in yet. [Fyodor] - -o Nmap UNIX distro build script should regenerate script.db. [Fyodor] - o Now it is in make prerelease - -o Nmap build system should be split into [Fyodor] - o prerelease -> generates version files, man pages, script.db - etc. That has to be done on one system, and then results checked in - before doing a make release. It does this stuff based on the - directory it is run in rather than some set dirname or a pure SVN - version - o release-tarballs -> does any system-dependent building and creates - the source tarballs. It does this stuff based on the directory it - is run in rather than some set dirname or a pure SVN version - o release-rpms -> Same as above, but also uses the created tarballs - to build the Linux RPM binaries for the current platform based on the - tarballs. - -o Build x86 and x86-64 VM instances for RPM building. [Fyodor] - * I think I'll use CentOS 5.3 - -o [NSE] Script scanning does not seem to work on Fyodor's Linux - machines after being installed from latest SVN (or 4.85BETA9) and run - as a non-root user (it works fine as root). The command "nmap -sC - localhost" leads to NSE failure messages which differ based on the - exact version run. [Was a relatively simple permissions problem in - our Makefile.in -- I fixed it] - -o [NSE] Release socket locks on connection failure or - timeout. [Patrick] - -o Update Nmap entry on Linux Online - - http://www.linux.org/apps/AppId_1979.html - - Screw it, the site does not seem to be maintained at all. They - aren't taking updates as of 6/2/09, and even Firefox shows latest - update as 0.9.1. - -o [Ncat] In verbose mode, print when an SSL connection is established - successfully and give the leaf certificate hash to make it easier to - verify when connecting to a machine where you can't or don't want to - use --ssl-verify (e.g. connecting to an ncat ssl server where it - created its own key). While we're at it, we might want to print - some other information from the leaf node, such as organizationName - and maybe localityName, countryName or something. We don't want to - be too verbose, but 1 line would be great and 2-3 might be - acceptable. [David] - -o Fix NSEdoc to better escape single-quotes in fields. If we can't do - that for some reason, we need to document it better. For example, - when we initially tried generating nsedoc for - http-webdav-unicode-bypass.nse, NSEdoc was listing it as a module - named "s auxiliary module", apparently because this line exited in - the description field: - This module is based on Metasplit's auxiliary module, modules/auxiliary/scanner/http/wmap_dir_webdav_unicode_bypass.rb. - (For full example, see scripts/http-webdav-unicode-bypass.nse - r13345) [David/SoC] - -o --script-args should allow a wider range of characters, and should - give a more useful error message if it receives chars it really - can't handle for some reason. For an example, try - "--script-args=smbuser=admin,smbpass=pass^word". For more details, - see Ron's report at - http://seclists.org/nmap-dev/2009/q2/0378.html. - -o [Ncat] Have --ssl-cert and --ssl-key send a certificate in connect - mode so that client certificate auth can be done. [David/Venkat] - -o Once we're done with host discovery empirical research, add it to - host-discovery.xml. Would be great to show the best combinations to - use for a given number of probes, the efficiency of the common probes - by themselves, etc. - -o Consider making the ping scan default be more comprehensive. Note - that I got 23% more Internet boxes found out of a 50K sample (see host - enumeration chapter of my book for details). Maybe I should - experiment a bit more to ensure they are real boxes and not network - artifacts and figure out exactly which tests are helping the most. - If I do this change, I'll have to update the host enumeration - chapter. For UDP probing purposes, we should test whether including - extra data in the packet (e.g. --data-length) helps in general, and - for services such as 53 and 137, we should probably send proper - protocol headers (e.g. a DNS server status message) so that we - receive responses from listening services. - -o We should probably check for a system Lua in a "lua5.1" directory - rather than just "lua", as Debian and also my Fedora 10 systems seem - to have that. See - http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=527997. [Note, - Fyodor asked the bug reporter Jan Nordholz on 5/14/09 if he could - write a patch. Jan sent in a patch, it worked, Fyodor checked it in.] - -o [NSE] Get rid of ceil so that floating point NSE runlevels work - again (some scripts, including (smb-brute) rely on this. They got - broken with the NSE core lua rewrite. [David]. - -o NSE script logical operator stuff is now documented in - scripting.xml--add to refguide.xml as well. [David/Patrick] - -o [NSE] Correct nsock_connect to unlock the socket slot if the - connection fails. When a socket is closed, it is unlocked so the - arbitrator can potentially open up a socket for another thread. But - Patrick discovered that a socket is not automatically unlocked when - a connection fails or times out, only when it is closed - explicitly. So that could hold up socket allocation for other - threads until garbage collection. May be a cause of slowness or - possibly deadlocks. [Patrick] - -o [NSE] Solve segfault issue which occurs when Nsock events call back - on a thread that has already ended (e.g. timeout, crash, early exit, - whatever) and been garbage collected. May want to just nsi_delete - all nsock sockets immediately upon thread ending. For an example of - this type of segfault, see - http://seclists.org/nmap-dev/2009/q2/0289.html. David says " I think - in the interests of getting this in a stable release, we should use - that strategy of closing all a thread's sockets. That ought to fix - all the problems above. Not to rule out a more thoughtful redesign - in the future." [David,Patrick] - -o We added the SEQ.CI value in Feb 2009 with 0 matchpoints. At some - point (once we have some real-life values) we need to evaluate whether - we want to give it points. A good time to do that would be when we - next do fingerprint integration, so we will actually have examples - of .CI in the nmap-os-db. [David] - -o [NSE] Make it a warning rather than error if a script in script.db - can't be found. [Patrick] - -o Add version detection signature for Ncat chat once we finalize the - announce format. [David] - -o Change Nmap signature files to use the .sig extension rather than - .gpg.txt, as that seems to be what gpg recommends. In fact, gpg - will automatically verify the right file if it exists after dropping - the .sig (or .asc) extension. I may need to configure .htaccess to - serve .sig files properly. Update nmap-install.xml - accordingly. Suggested by tic at eternalrealm.net by email on - 7/13/08. [Fyodor] - * Rename existing files, add symlink from the old .gpg.txt to .asc - versions - * Add appropriate .htaccess content type if needed for downloads - - not needed since I decided on .asc extension rather than .sig - * Update the generation scripts - * Update the book documentation - - https://nmap.org/book/install.html#inst-integrity - -o Ask Coverity if they'll scan latest version of Nmap. [Fyodor asked - David Maxwell on 5/14/09 ] - -o Make 4.85BETA9 release [Fyodor] - -o [Zenmap] Make a way to start a scan from the profile editor without - creating a profile, then remove the command wizard. This is partial - implementation of - http://www.bamsoftware.com/wiki/Nmap/ZenmapCommandLine. [David] - -o [Ncat] Make proxy server mode work on Windows (this is the last - remaining fork() dependency in Ncat). - -o Do an OS detection integration run -- last was based on - 1/8/09. [David] - -o [Ncat] Maybe we should create an SSL cert with no passphrase during - Ncat compilation or install process so that if someone specifies - Ncat -l and --ssl with no --ssl-cert and --ssl-key, we already have - one for them, and it is a slightly better one (since the private key - isn't known) than if we distributed a key. Obviously it is still - subject to MITM attacks since there is no domain validation going - on. But people who need that will have to buy a key from a - certificate authority in any case. We could create the key by using - the "openssl" command line tool as shown in - https://nmap.org/ncat/guide/ncat-advanced.html#ncat-ssl, or maybe - better to have a way for ncat to do it using openssl calls. [David] - -o [Zenmap] Should probably give some sort of widget indication that a - scan is running. Now that we can start multiple scans at once, the - "scan" button goes back to being unpressed while the scan is - running. As some scans take minutes or more to show output, it is - not always clear whether they are still properly running. We should - probably have some sort of widget, such as the throbber used in web - browsers, to show that Nmap is still running. It could be fore a - specific scan (kind of like how you have a separate throbber for - each tab on a web browser), or a global one which means at least one - scan is running. Or maybe a different sort of indication is in - order (like a timer). [David] - -o Further investigate Nmap Proxy patch by Zoltan Panczel and Ferenc - Spala. See http://nmap-dev.fw.hu/ and - http://seclists.org/nmap-dev/2009/q1/0255.html . [Discussed it and - then added new proxy feature item] - -o Wherever practical, fix compiler warnings when compiling Nmap with - VC++ 2008 Express SP1 (there aren't many). [David] - -o [NSE] Consider adding boolean expressions to --script arguments. For - example, see Patrick's implementation at - http://seclists.org/nmap-dev/2008/q3/0300.html . - -o Generate a list of trusted SSL certificates to ship with Ncat (by - extracting f rom Mozilla or similar), and install them with - Ncat. Decide how these certificat es should be preferred to any - system-provided certs, if any. [David] - -o [NSE] Add desired SoC09 infrastructure ideas to this TODO to the - extent they don't already exist. - -o [Ncat] Consider supporting server certificate verification when used - in client SSL mode. - o For now we document in user's guide that it is not secure. - o Maybe we can do an ssh-style approach where we just print the - fingerprint and expect the ncat client user to ensure it is the - right one? - o If we're going to verify cert's etc., we need to also make sure we - are actually using secure ciphers. We may need to update nsock to - support cipher selection, because we want fast ones for version - detection, but usually want secure ones for NSE and/or ncat. - o Do we want to check all this by default, or offer an option for - it? Doing it by default is more secure, though it can be annoying - when a certificate has expired, is self-signed, you connect to - domain.com when the certificate is for www.domain.com, etc. If it - is done by deault, we might just print an error message. Whreas - if we have a special option, it may be OK to exit and refuse the - connection. - o What certs should we allow? Same as the browsers do? Maybe get - rid of Comodo? Maybe we should fail to recognize any certs with MD5 - in the trust chain? - o What about people who are running their own SSL service and just - want to specify the cert file they use, because they generated it - themself and not from a trusted CA. - o Need to check expiration, domain, etc. if we're checking certs at - all. - o We can probably get away with not doing revocation checking, as - long as we document that we don't. - -o consider changing status field from "up" and "down" to "online" and - "offline". Actually, maybe we don't want this after all. - online/offline look pretty similar, and they're longer too. I'm - taking this out of the TODO. - -o [Ncat] When acting as an HTTP proxy, we should support GET mode as - well as CONNECT so that it works as a non-SSL proxy in browsers such - as firefox. [David] - -o Finalize GSoC applicant research, communication, and selection - [David, Fyodor] - -o Go through all the SoC applicants and decide who we want to accept - and start communicating with them. [David,Fyodor] - o Decide which applicants we want, and who would be best for - mentoring them. - -o Document that U1.RID gives "G" as long as all the data bytes in the - echoed response data are "C" as expected. This G code is still - given even when the response is truncated, including if there are 0 - bytes echoed. [David] - -o [Ndiff] Rethink the output format. David says: In particular, I - would like to always have the old state on the left and the new - state on the right: "was filtered, is open," not "is open, was - filtered." I also like the context diff output of MadHat's - nmap-diff. [David] - - -o Canonicalize the "host up" messages for port scan and ping scan so - that instead of things like "Host scanme.nmap.org (64.13.134.52) - appears to be up ... good." we standardize in both cases on - something like: "Host scanme.nmap.org (64.13.134.52) is up (.75s - latency)". Note the addition of the latency value, which is our - srtt value for the host. This will only show in ping scan and - verbose port scan because the line doesn't appear without verbose - mode. [David] - -o Ping scans always seem to say "0 [hosts] undergoing Ping Scan" when - you request stats, rather than the proper number. For an example, - try a command such as "nmap -iR 10000 -sP -n" and then press enter - during the scan. Here are some examples of the bad output: Stats: - 25:34:33 elapsed; 991232 hosts completed (72530 up), 0 undergoing - Ping Scan Ping Scan Timing: About 53.69% done; ETC: 22:49 (0:00:09 - remaining) Stats: 0:01:10 elapsed; 0 hosts completed (0 up), 0 - undergoing Ping Scan Ping Scan Timing: About 24.03% done; ETC: 22:42 - (0:03:41 remaining) Stats: 0:03:28 elapsed; 4096 hosts completed - (284 up), 0 undergoing Ping Scan Ping Scan Timing: About 3.06% done; - ETC: 22:44 (0:03:07 remaining) [David] - - -o Remove obsolete tests from nmap-os-db itself. [David] - -o Prepare for Summer of Code - * Brainstorm for ideas - * Create new ideas page - * Apply to participate in program again - * Advertise for applicants - * Evaluate applicants - -o NSEDoc script/module documentation pages should probably provide a - link to the script/module source code (except for C modules). The - link format should probably be of the form - https://nmap.org/data/scripts/[script].nse and - /data/nselib/[module].lua. NSEdoc can assume they already exist - there, as we'll probably put them there using the same system we use - to copy other stuff to the data dir. - -o [Ncat] Let people set up authenticated proxies using - --listen and --proxy-auth together (right now we don't support - that). [David] - -o When you specify multiple comma-separated arguments to --script, - those arguments seem to get lost when the Nmap command is printed in - Nmap's output files. For example, I run the command: - nmap -oN - --script=discovery,intrusive scanme.nmap.org - The output includes: - # Nmap 4.85BETA4 scan initiated Thu Mar 26 15:40:05 2009 as: ./nmap - -oN - --script=discovery scanme.nmap.org - Note the missing ",intrusive" in the script argument. [David] - -o Merge patrick/nse-lua-merge for easier-to-maintain and simpler - codebase once David and Patrick are happy with it. [David] - -o SVN check out /nmap as an external in a directory named svn or src - or nmapsvn or something under nmap.org web tree. Then redirect the - individual nmap.org/data/ files, where needed, to the nmapsvn - instead. and update nmap-dev Makefile not to copy them to the - /data/ dir anymore. Then update the nsedoc system to generate proper - links to the new script/nselib locations. [Fyodor] - -o Improvements to presentation of version detection - information. [Brandon] - o Allow longer strings. Right now it can be 128 chars for the - fullversion info, I think. But that isn't enough for this useful - information-packed string: "Apache httpd 2.0.52 ((Red Hat) - mod_perl/1.99_16 Perl/v5.8.5 DAV/2 mod_jk/1.2.19 PHP/4.3.9 - mod_python/3.1.3 Python/2.3.4 mod_ssl/2.0.52 OpenSSL/0.9.7a)". - After discussion w/Brandon, we're going to allow 160 chars total. - o Instead of omitting all information when version info string too - long, we're going to truncate and allow 157 characters, plus - ellipses (...) - o Brandon says: "my final gripe is that the full version string is - constructed as (). - but, even if product or version are blank, the spaces are still - there" - -o I need an output-autoflush option of some sort. This could be - useful to ensure I get all the --packet_trace and debug data before - Nmap crashes. Actually, I'm not sure that is so critical. - o Killing it for now, not sure that it even is needed. - -o Fix the directory function(s) in nse_fs.cc to be usable by scripts and - improve flexibility. [this entry added by Patrick] - -o [Ncat] The sys_wrap.c/.h code contains a whole bunch of capitalized - versions of system calls (Fork(), Socket(), Sscanf(), etc.) which - are mostly the same as the standard version except that they cause - ncat to quit if they are triggered. They also may be used partially - for portability. The main issues are: - 1) Because the function quits in the case of errors, it doesn't - always have the context to print a useful error message (and - even when it does, it often doesn't -- for example Fopen could - print the filename, but doesn't.) Also, sometimes these - functions are called when quitting really isn't the desired - outcome of an error. - 2) Some could be replaced by code in nbase, for example, Malloc - basically does the same thing as our safe_malloc already used - throughout Nmap. - So we should probably consider simplifying/removing this code to the - extent possible. But we need to remember to add error detection to - the callers where necessary rather than blindly switching from - (e.g.) Connect() to connect(). [Kris or David] - -o With --version-trace (may be a problem with other uses of nsock - tracing too), I often get dozens of "wait_for_events" reports in a - row in a very short period, flooding the logs. For example, with - the command "nmap -sV --version-trace www.google.com", I get: - NSOCK (22.3570s) Callback: WRITE SUCCESS for EID 283 [74.125.19.147:443] - NSOCK (22.3570s) msevent_delete (IOD #4) (EID #283) - NSOCK (22.3570s) wait_for_events - NSOCK (22.3570s) wait_for_events - NSOCK (22.3570s) wait_for_events - NSOCK (22.3570s) wait_for_events - NSOCK (22.3570s) wait_for_events - NSOCK (22.3570s) wait_for_events - NSOCK (22.3570s) wait_for_events - NSOCK (22.3570s) wait_for_events - NSOCK (22.3570s) wait_for_events - [Goes on for pages] - -o NSE memory issues (and gh_list assert failure) [David] - o See this thread: http://seclists.org/nmap-dev/2009/q1/0532.html - o We're taking this out for now since the new nse-lua-merge - tenatively looks like it fixes this. - -o [Ncat] Why does Ncat require enclosure in a while loop to answer - repeated UDP queries, but not TCP? For example, see the "Emulating - Diagnostic Services" section of the Ncat user's guide. - o Note: http://seclists.org/nmap-dev/2009/q1/0133.html - -o Determine what we should do about the IE.DLI OS detection test [David] - o All of the 1656 results for this test in nmap-os-db are DLI=S. - o Is the test not working right (producing the proper results - against targets), or is it just a generally useless test for - which virtually all targets respond the same way? - o Are there other "useless" tests in nmap-os-db? It is worth - checking, IMHO. - o We're going to get rid of IE.DLI, IE.SI, U1.RUL, and maybe TOS and - TOSI tests. - -o When you do ncat -h, Ncat should probably show the Nmap version - number rather than (currently) 0.2. Also ncat in -v mode should - show that same header. [David] - -o Ncat verbose mode (-v) should probably only give important messages, - such as perhaps a message once you connect successfully to a port, - or a message if the connection attempt times out. An Ncat version - banner (with URL) like Nmap has might be warranted (in verbose - mode). Currently, Ncat floods you with (mostly) useless debugging - information like this with a single -v (this output, on the other - hand, might be useful for a debugging option): [David] - # ncat -C -v scanme.nmap.org 80 - NSOCK (0.0000s) TCP connection requested to 64.13.134.52:80 (IOD #1) EID 8 - NSOCK (0.0200s) Callback: CONNECT SUCCESS for EID 8 [64.13.134.52:80] - NSOCK (0.0200s) Read request from IOD #1 [64.13.134.52:80] (timeout: -1ms) EID 18 - NSOCK (0.0200s) Read request for 0 bytes from IOD #2 (peer unspecified) EID 26 - GET / HTTP/1.0 - NSOCK (4.4280s) Callback READ SUCCESS for EID 26 (peer unspecified) (15 bytes) - NSOCK (4.4280s) Write request for 16 bytes to IOD #1 EID 35 [64.13.134.52:80] - NSOCK (4.4280s) Callback: WRITE SUCCESS for EID 35 [64.13.134.52:80] - NSOCK (4.4280s) Read request for 0 bytes from IOD #2 (peer unspecified) EID 42 - For comparison, here is what Eric Jackson's nc (The nc available in - Fedora 10's package repository) shows in verbose mode for the same - connection: - # nc -v scanme.nmap.org 80 - Connection to scanme.nmap.org 80 port [tcp/http] succeeded! - GET / HTTP/1.0 [David] - -o Final polishing of our GSoC pages. [Fyodor] - -o Advertise widely for Nmap GSoC applicants [Fyodor] - -o [Ncat] We should (maybe) consider a way for people to choose - usernames in --chat. - o Removing this for now. We can add it back if we decide we really - want this. - -o Deal with new Python 2.6 Zenmap build warnings: - C:\Python26\lib\site-packages\py2exe\build_exe.py:16: DeprecationWarning: the sets module is deprecated - import sets - http://sourceforge.net/tracker/index.php?func=detail&aid=2314799&group_id=15583&atid=115583 - [Bug in py2exe, will probably be fixed with a new version of py2exe - once it is released and we upgrade. This isn't causing us any major - problem anyway.] - -o When I scan large groups of hosts with OS detection enabled, I get - groups of warnings like: - Insufficient responses for TCP sequencing (0), OS detection may be less accurate - Insufficient responses for TCP sequencing (0), OS detection may be less accurate - Insufficient responses for TCP sequencing (0), OS detection may be less accurate - Insufficient responses for TCP sequencing (0), OS detection may be less accurate - Insufficient responses for TCP sequencing (0), OS detection may be less accurate - Note how it doesn't even tell the relevant IP address, and it isn't - included in an individual host section. We should probably either - include it in the section for an individual host, like we do with - "OSScan results may be unreliable because we could not find at least - 1 open and 1 closed port", or (not quite as - good) include the relevant IP address in the error message. And we - may or may not want to require verbose mode. - -o Ncat chat should bomine the "already connected" user list into one - line, like: - already connected: 69.232.238.42 is connected as , 206.81.65.43 as , 69.232.238.42 as - -o [Ndiff] Maybe Ndiff should display changes to version detection and - OS detection information? [David] - o Version detection done, now just needs OS detection. - -o When I start ncat chat with this tcsh command: - ncat -l --chat scanme.nmap.org < /dev/null >& /dev/null & - The first client to connect to the chat becomes user0 and doesn't - work quite right. Messages user0 type get transmitted to other - clients, but user0 does not see their messages. Nore does user0 get - the normal connection announcement upon connecting. If I quit - user0, the next client to connect becomes user0 again and has the - same problem. If I start ncat on the server with "ncat -l --chat - scanme.nmap.org" (no redirection), other clients can connect with no problems. - -o Ncat --chat should probably announce to everyone (including the new - person) when someone connects. This tells the new person their - username, and lets everyone else know about the new connection. [David] - o We should also tell the new person (and possibly everyone on the - channel) the list of existing participants. - -o SoC ideas page [Fyodor] - -o Nmap 4.85BETA4 release [Fyodor] - -o [Ncat] Wouldn't it be nice if we could support --exec (and maybe - some sort of partial-emulated --sh-exec) on Windows? [David] - o Almost working! We found some problems with "ncat.exe -v -l - --sh-exec "ncat -v scanme.nmap.org" - -o [Ncat] Can we use it as an IPv4 <-> IPv6 gateway? If so (or if we - can add it), it should be added to the ncat guide feature list. - o Yes, David tried it with --sh-exec and it worked. - -o [Ncat] We should probably make it work without OpenSSL. When I try - ./configure --without-openssl on latest svn Nmap, Ncat build fails - with: - gcc -MM -I../libpcap -DHAVE_CONFIG_H -D_FORTIFY_SOURCE=2 -I. -I.. -I../nsock/include/ -I../nbase ncat_main.c ncat_connect.c ncat_core.c ncat_listen.c ncat_proxy.c ncat_broker.c ncat_hostmatch.c ncat_ssl.c util.c sys_wrap.c > makefile.dep - make[2]: Leaving directory `/mondo/fyodor/nmap/ncat' - make[2]: Entering directory `/mondo/fyodor/nmap/ncat' - gcc -I../libpcap -DHAVE_CONFIG_H -D_FORTIFY_SOURCE=2 -I. -I.. -I../nsock/include/ -I../nbase -c ncat_main.c -o ncat_main.o - ncat_main.c: In function ‘main’: - ncat_main.c:536: error: ‘struct options’ has no member named ‘ssl’ - ncat_main.c: In function ‘ncat_listen_mode’: - ncat_main.c:646: error: ‘struct options’ has no member named ‘ssl’ - ncat_main.c:646: error: ‘struct options’ has no member named ‘sslcert’ - ncat_main.c:646: error: ‘struct options’ has no member named ‘sslkey’ - make[2]: *** [ncat_main.o] Error 1 - make[2]: Leaving directory `/mondo/fyodor/nmap/ncat' - make[1]: *** [build-ncat] Error 2 - make[1]: Leaving directory `/mondo/fyodor/nmap' - make: *** [static] Error 2 - -o [Ncat] Defensive coding review of Ncat --chat (talk) - -o [Ncat] As SSL server it should not crash when someone connects in - w/o SSL and does ^C. When David tried it during our chat, the ncat - servr "ncat --broker --ssl-key test-key.pem --ssl-cert test-cert.pem - --ssl --chat -l" crashed with: SSL_accept(): - error:00000000:lib(0):func(0):reason(0). Also, when a Windows SSL - clients joined and then left, the server died with "Broken pipe - -o [Ncat] --chat should probably only allow reasonable chars, to avoid - cntrl-chars, etc. - -o Nmap should treat ports named "unknown" in nmap-services the same - way (from a naming perspective) as it treats ports which are not - listed at all. See http://seclists.org/nmap-dev/2009/q1/0589.html. - -o Ncat user guide "Emulating Diagnostic Services" page has a very long - UDP chargen server line which causes wrapping problems in web browsers - (e.g. it widens the page substantially). It should probably be - split into multiple lines. [David] - -o Ncat user guide proxying section says "The only exception is when - listing a proxy host by IPv6 address; then the port is required." - Why would we require a port number for IPv6 rather than just use the - same defaults as we do for IPv4? - [David explained that this is because to do otherwise would be - ambiguous because IPv6 uses : for separaters, so we wouldn't know - how to handle things like FF::10:80] - -o [Ncat] Perhaps we should make --ssl work in --chat. If nothing - else, it might be useful if you want to reduce the number of people - connecting with telnet, etc. rather than ncat. - -o [Ncat] --talk should probably be changed (in the code and - documentation) to --chat, as Ncat chat has a - much nicer ring to it, IMHO. --talk should remain as an alias to - --chat, but we don't need to document it. [David] - -o Ncat Windows issue where you make a connection and then take several - seconds to type in a line to the server, Ncat wrongly times out when - trying to write your line to the remote server. [David] - -o Ncat write timeout problems cause client to quit due to write - timeout sometimes. [David] - Examples: - o yes | ncat localhost - o when we paste a few lines into the terminal window in an Ncat chat - -o Defensive coding review of ncat_proxy.* [David] - -o Process the latest version detection submissions. We now have more - than 1,700 of them queued up. [Doug] - -o Write Ncat users' guide, demonstrating all the neat stuff you can do - with it. This should probably be in DocBook XML so it can be an NNS - chapter. You might want to query nmap-dev for list of neat things - people do with ncat (or look around for what people do with nc). - Testing it out for examples might expose areas for improvement as - well. [David] - -o Look at Dario Ciccarone's email from 5/1/07 about IPID sequence - issues, and consider adding IPID sequence test for closed-port-tcp as - they apparently can be different. [David] - o Also fix bug which causes SEQ to not be printed if the TCP open - port tests fail to produce results, even though the II and - (upcoming) CI tests may have useful results. [David] - -o NSE should offer some way to sleep/yield for a given amount of - time. This would allow other scripts to run while a script has - nothing to do. Possible uses: - o Many services have rate limits (or you might just want to use them - for politeness). For example, a web site spidering application - might want to limit HTTP requests to some number per second to avoid - pissing off the target webmaster more than is necessary (or prevent - getting auto-blocked). Similarly, whois servers often will block - IPs which query them too often in a short period. Or maybe you - don't want to exceed the threshold limits of an IDS. - o Example current scripts which might benefit: sql-injection, whois - (possibly), pop3-brute, etc. - o If we don't currently have a way for a cpu-bound NSE script to - yield, then perhaps this could help us implement such a mechanism. - But maybe coroutine.yield already does the trick. - o The mechanism needs to be documented, and ideally should be - implemented in at least one of the scripts shipped with Nmap. - -o Consider adding a way for requesting timing status updates at a - given interval (such as every 5 seconds) to XML and/or normal - output. This would be useful for people who run Nmap from scripts - or other higher level applications. [David] - -o Ncat --allow/--deny bug: "--allow and --deny only support host - specification by IP address, and give no warning when you use - another form such as a host name." Should probably use same syntax - as --exclude. We also want to at least do verification at the - beginning to make sure all the entries are legitimately formed. We - probably want to do things like DNS resolution at the beginning - too. Otherwise we might have a DNS failure when we actually get a - connection and perhaps have to reject the connection wrongly, or - risk a false negative. [David] - -o Fix this overflow: - Stats: 93:57:40 elapsed; 254868 hosts completed (2048 up), 2048 undergoing UDP Scan - UDP Scan Timing: About 11.34% done; ETC: 03:21 (-688:-41:-48 remaining) - [Done by David and Henri Doreau] - -o Ncat -- perhaps connection brokering should support UDP as well as - (its existing support for) TCP? Actually this does raise issues - such as deciding what list of UDP systems to forward a packet too. - Its obviously not like TCP where you have a list of open - connections. Ncat could build such a list, but, for example, would - never know when to remove the host. For now, David is just going to - adjust the error message to encourage people to email nmap-dev - describing their usage scenario if they want this feature. - -o Ncat documentation should note that no SSL certificate verification - is done (maybe we should offer an option to do so, if OpenSSL makes - that easy). - o Done in the new Ncat user's guide - -o Fix dns-zone-transfer infinite recursion bug described at - http://seclists.org/nmap-dev/2009/q1/0317.html. It sounds like the - best approach is to use our dns.lua library rather than having - dns-zone-transfer do its own DNS packet parsing. - -o Fix XML escaping issue so that improper chars from NSE scripts or - elsewhere can't cause corrupt XML files. See - http://seclists.org/nmap-dev/2009/q1/0316.html for an example. [David] - -o Look into whether we should increase the frequency of port scan - pings. See http://seclists.org/nmap-dev/2008/q1/0096.html . Note - that Fyodor already increased them a bit in 2008. Might not need - more. [David did extensive testing of this one already] - -o Find way to document NSE library script arguments and perhaps have - them bubble up to scripts themselves. For example, I had to read - the SNMP library source code to determine the script argument to - specify the SNMP community name for snmp-sysdescr - (https://nmap.org/nsedoc/scripts/snmp-sysdescr.html). Maybe we could - just standardize on something like we do with SMB library and the - scripts which call it (https://nmap.org/nsedoc/modules/smb.html, - https://nmap.org/nsedoc/scripts/smb-check-vulns.html). [David] - -o If it wouldn't bloat things too much, it would be nice to include - ndiff in the Nmap win32 zip distribution files. - -o Reported NSE crash: - "Assertion failed - file ..\nse_main.cc line 314 - lua_gettop(L_script_scan) == 0" - o He says: "After looking at this closer, it appears the assertion - occurs if I include the IP where the scan is run from. For us, I'm - running this on IP 57, which is a VMware Windows Server image. If - I eliminate that IP from the range it successfully completed the - scan for all other devices." - o Seems to be fixed. He can no longer reproduce the problem with - 4.85BETA3. - -o Deal with GTK DLL problem with Nmap 4.85BETA1: [Fyodor] - o David's installer seems to work--he's using a different GTK - distribution. I'll try that. Works! Done! - o Details on problem: http://seclists.org/nmap-dev/2009/q1/0207.html - o Quick workaround done for 4.85BETA2, but better solution needed. - -o "SCRIPT ENGINE (250.600s): ./scripts/rpcinfo.nse against - a.b.c.d: ended with error: ./nselib/datafiles.lua:114: attempt - to index global 'arg' (a nil value)" - -- http://seclists.org/nmap-dev/2009/q1/0227.html [Patrick] - -o Consider making the TODO list public - o Done: http://seclists.org/nmap-dev/2009/q1/0175.html - o Probably remove all of the "done" items since that is easier than - reviewing them. - o Might as well add to insecure.org/nmap/data/ - o Maybe a bug tracker is a better approach. - -o [NPING] Fix compilation on Solaris. See - http://seclists.org/nmap-dev/2010/q1/870. - diff --git a/todo/gorjan.txt b/todo/gorjan.txt deleted file mode 100644 index 3eada0924..000000000 --- a/todo/gorjan.txt +++ /dev/null @@ -1,66 +0,0 @@ -===== -GSoC 2011 participation: Discovery and miscelaneous script specialist -===== - -Work in progress: - -* bgpmon-info analyze - -* bittorrent-dht-nodes - -* lldp - write script proposal -http://en.wikipedia.org/wiki/Link_Layer_Discovery_Protocol - -* disjunctive-traceroute analyze feasibility -http://ccr.sigcomm.org/online/?q=node/398 - -===== - -ToDo: - -* snmp-brute port to brute framework -There are a couple of default passwords that snmp-brute uses atm which should be -considered even when it's the brute.lua is used - -===== - -Maybe (the ones with ** aren't on the Script_Ideas Page yet) - -* Bonjour / mdns / llmnr etc. -(DNS protocols support) + backscatter into dns scripts where applicable? - -* targets-asn -John Bond is working on this. It's called asn-to-prefixes. Perhaps I could -review it, asist so it makes its way to the library faster? On the other hand -there already are a couple of people assisting. - -* targets-dhcp -dhcp-discover as a prerule, so it doesn't run by default. But it doesn't run by -default. It's discovery, intrusive, but not default. Maybe just add the prerule -there, and some way of forcibly initiating the prerule (like an argument). - -* hnap-info -* hnap-auth-bypass -A nice hnap library would be fitting, that will make these scripts a breeze. -I'd need testing equipment, or some :S implementation. - -* vuze-dht-version -* Nbstat.nse -> change to using a broadcast prerule -* SSL renegotiation -* soap.lua -* xmlrpc.lua - -===== - -Completed: - -* broadcast-ping -* nmap lib: get_ttl() and get_payload_info() -* ip-geolocation scripts -* snmp-interfaces patch related to mac-geolocation -* mac-geolocation -* stdnse.lua: in_port_range() -* backorifice-brute -* backorifice-info - -===== \ No newline at end of file diff --git a/todo/henri.txt b/todo/henri.txt deleted file mode 100644 index cca881455..000000000 --- a/todo/henri.txt +++ /dev/null @@ -1,41 +0,0 @@ -o Proper SSL support in proxy mode. - - A naive implementation relying on the current code would probably look - horrible (at least my own attempts did). I believe that nsock should - internally be able to SSLify a plain TCP connection. It doesn't have to be - exported but it should be implemented just like the other operations. Then - it would be trivial (and clean) for the library to SSLify the channel - established by the proxy hooks. - - When redesigning nsock SSL code, keep in mind the ability to establish a SSL - session and still expose the raw TCP. That can be convenient when auditing - the SSL/TLS layer. -o Don't drop pending writes when deleting the corresponding IOD. For nsock to - behave a bit like standard BSD sockets we should flush writes on close. (OTOH - anything which isn't ack'ed has no meaning, caller can still cancel it - typically...) -o Give IODs their own methods to streamline the code and get rid of all - the special cases in nsock_core.c. This would also make it easier to - hook operations (typically: override the default iod_connect() method - to establish a proxy chain). -o Fix the read API (!) -o Profile the pcap code. It needs cleanup (for sure) and optimizations (maybe). -o Proxy authentication -o Handle socks4a - - This requires to figure out how to trigger proxy code without - resolving target hostname first. The problem is that the proxy code - is supposed to be a transparent hook of connect()... Extending the - exported API will probably be needed :( - - Async hostname resolution available from within nsock would let us - try clever tricks... I'm not sure whether nsock should provide it - or if it should simply provide an API to plug an external system. -o Socks5 support -o Some code is copied from ncat. I should move it to nbase. -o Replace event lists by more efficient data structures. Consider using - a radix tree to map event IDs to pointers. Another solution would - be to put them all into a single RB-tree (TODO: validate BSD_HACK_MODE - & stuff). Encoding the event type in the ID's MSB would let us do inorder - traversal with connect events first, then read, then write... - {NOTE: It'd be cool for the beauty of it, but my tests reveal that as of Oct. - 2013 there's no big bottleneck there.} -o Rework the filespace code to avoid unneeded data copy. Scatter/gather - I/O might be useful there. Same task can also be expressed as: "profile and - optimize the usual nmap nsock I/O patterns." diff --git a/todo/nmap.txt b/todo/nmap.txt deleted file mode 100644 index 76fe1ff19..000000000 --- a/todo/nmap.txt +++ /dev/null @@ -1,638 +0,0 @@ -TODO $Id: TODO 11866 2009-01-24 23:10:05Z fyodor $ -*-text-*- - -o Work on Nmap on Mobile devices, particularly Android. Would be - great to get it in Google Play store, for example. An official - version with a workable GUI. For now, people have to do manual work - and it isn't as well tested either: - https://secwiki.org/w/Nmap/Android . If this is successful, we could - consider iOS. - -o Nmap performance work. Particularly with --min-rate. - -o Consider re-architecting Nmap to have more of a scanning pipeline -approach rather than fixed sets of hosts which start and finish one -phase and then move into the next in parallel. This could potentially -allow us to add hosts one by one to a phase as other hosts finish that -phase and, ideally, the phases could run in parallel too. - -o Nmap Network Scanning, 2nd Edition work [placeholder] - -o Organize nselib into a hierarchy. A library "dirname/filename.lua" can be - required as "dirname.filename". We would need to ensure the installers - (Makefile, OS X, Windows, RPM) can handle this. See - http://seclists.org/nmap-dev/2014/q3/364 - -o We should work to reduce Zenmap's memory consumption. We used to - commonly get error reports from people who load so many systems that - Zenmap gives an out of memory error and crashes. For example, see - this thread: http://seclists.org/nmap-dev/2014/q2/46 - After committing patch at http://seclists.org/nmap-dev/2014/q2/429, - we no longer get the error report but the problem still exists. - The problem seems to lie in a very large Nmap Output being stored - in memory and a possible fix seems to be to use a file based paging - system. - -o Consider making a version of Nmap for Apple's official Mac App - Store. A particular concern with the downloadable Mac version of - Nmap is that Apple's new "Mountain Lion" release may require users - to jump through hoops to install unsigned non-app-store content per - their "Gatekeeper" "feature". Though maybe signing the app will be - enough. There may also be an issue with the "Sandboxing" - requirement for App Store apps starting June 2012. Will Nmap be - able to request all the permissions it needs? Ignoring the - technical challenges for the moment, what will users prefer? - -o Do a roll up on (state, TTL) pair instead of just state so that TTL - info is not lost when doing roll up on port states. - See thread at http://seclists.org/nmap-dev/2014/q3/93 - -o Consider looking into differring TTL values during OS detection - phase and choose a port that is (hopefully) not firewalled to get - a better chance at correct result. See thread at - http://seclists.org/nmap-dev/2014/q3/33 - -o [Zenmap] Look into and refactor code which uses the (very slow) += operation - on strings. http://seclists.org/nmap-dev/2014/q2/432 helped improve speeds - for opening files (from hours to seconds) and it seems like more speedups - can be done in other places. - -o Look into moving our Mac building/testing system into a virtual - machine or leased server sort of environment so that multiple Nmap - developers can access it and nobody has to keep a stack of Mac Minis - in their closet. - -o INFRASTRUCTURE: Upgrade seclists to use Mailman 3, which apparently - has many improvements. - -o We should fix nsedoc generation so it doesn't fail when blocks like - @usage, @output, etc. are followed by a local declaration. See - http://seclists.org/nmap-dev/2014/q2/331. If for some reason this - just can't be fixed, we will have to document the heck out of it, I - suppose. - -o When scanning your own IP from Windows, Nmap currently recognizes - the problem (can't do a raw scan like that on Windows) and skips the - SYN scan, leading to Nmap printing a bunch of ports in "unknown" - state at the end. Nmap should probably act like unprivileged mode - in this case (e.g. do a connect scan, etc.). See - http://seclists.org/nmap-dev/2013/q3/519 - -o Investigate Checkmarx static analysis report of Nmap source tree - that someone sent us on Feb 12. It looks like mostly false positives, - but we should go through to check for any real bugs or even possible - security issues. Fyodor has the report. - -o INFRASTRUCTURE: Consider updating our svn-mailer.py (and conf file) - to the latest official version. First check whether there is a - later official version and whether it has material changes. We're - currently using one from - subversion-1.4.2/tools/hook-scripts/mailer/mailer.py. - -o Consider a two-stage model for IPv6 subnet/pattern support - o Right now you can try to scan a /64, for example, and Nmap will try - to iterate through them all (and of course never complete). So - perhaps Nmap should first look at a specification and decide if it - should use other techniques like multicast discovery instead. - -o Move advanced IPv6 host discovery features from NSE into core Nmap. - We'll probably add the functionality of - targets-ipv6-multicast-invalid-dst, targets-ipv6-multicast-echo, and - maybe targets-ipv6-multicast-slaac. - - The idea is that Nmap does them automatically if it gets a large - target specification and sees that it is local so can be multicast - pinged. - -o We should figure out why (at least with Nping) raw ethernet frame - sends seem to be taking significantly longer than raw socket sends - (e.g. using --send-ip or the OS-provided ping utility). This has - been reproduced on Linux and Windows. Here's a thread: - http://seclists.org/nmap-dev/2012/q4/424 - o Note that David and I tried to reproduce this on his machine and - on 'web' and 'research' machines and could not reproduce. Still - happens with Fyodor's machine connected with WiFi. Fyodor should - test on the same machine using wired and see if that changes anything. - -o Implement some improvements to dns-ip6-arpa.nse, as describe at - http://seclists.org/nmap-dev/2012/q2/45. - - Also consider a move to "fire and forget" logic. Just blast out - the queries that we know we have to make, and then read any replies - that may happen to come back. (but still try not to introduce - inaccuracy (missed hosts) by flooding the network. - -o Treat the input to the escape function in xml.cc as UTF-8, not just - ASCII. Good UTF-8 should survive into the output; i.e., "\xe2\x98\xbb" - should become "\xe2\x98\xbb" in the output, not "☻". - If the input happens not to be UTF-8, (like the file name in - http://seclists.org/nmap-dev/2013/q1/180), I suppose we can - individually encode each byte of each invalid sequence: "\xba\xda\xbf" - becomes "ºÚ¿". Can probably do this with simple - byte->rune and rune->byte functions as in - http://plan9.bell-labs.com/sys/doc/utf.html. - -o We should probably redo the Nmap header (e.g. on https://nmap.org) to - make it more attractive. Or, at a minimum we should update the - screenshots and think about which links we really need (some of those - pages aren't really updated any more). - -o Test a hierarchical classifier for IPv6 OS detection. Our classifier - currently treats, for example, some localhost Linux fingerprints as - separate classes from remote Linux fingerprints, simply because we - lose precision if we lump them together (for example TCP window size - differs across certain Linux versions when measured remotely, but - not on localhost). This leads to the linear classifier having to use - narrow margins between fingerprints that are really very similar. I - want to try a tree of classification where each non-leaf node is a - separately trained classifier and each leaf node is a final - classification. The first layer of the hierarchy would be something - like - (linux windows solaris aix ... other) - where "linux" would contain *all* the Linux fingerprints in a single - class. Lower levels would be like - (linux-2.4 linux-2.6) - (windows-xp windows-vista windows-7) - Lower levels will include only those fingerprints in their parent - class, so we don't even think about Windows when classifying - Linux. Probably three or four levels will be sufficient. There may - be a principled or automatic way to build this hierarchy, but I - suspect playing it by ear will be sufficient. Talk to David for more - of his thinking on this topic. - -o Maybe we should rename dns-brute to dns-brute-enum since it is so different - from our traditional brute force authentication cracking -brute scripts? - -o NSE WORK (note that this is mostly infrastructure because script - ideas are generally put on the script ideas page instead: - https://secwiki.org/w/Nmap_Script_Ideas) - o Review NSE-based port scanning and RST idle scan. - http://seclists.org/nmap-dev/2011/q2/307. [Henri and Hani?] - -o Maybe we should add an analysis or reporting or intelligence (or - different name) for our NSE scripts which don't send any packets, but - simply analyze Nmap's existing data and report when useful. - -o Install some sort of svnview webapp for svn.nmap.org which is - wrapped in Insecure chrome, allows people to click link for direct - file download, probably shows revision history and allows users to - see older versions, etc. - -o Process Nmap survey and send out results [Fyodor] - -o Nping (we think) will stop after 2^32 rounds even when "-c 0" is - given. We should probably make this a 64-bit integrer so that "-c - 0" will go essentially forever and so that users can give values - higher than 4 billion. - -o Nscan work [placeholder] - - Hosted Nmap system - -o Add CPE entries to OS fingerpting DB entries which still lack them. - This is a gradual process since almost all of the missing ones - aren't in the official CPE dictionary either and it can take a lot - of research to decide on an appropriate entry. Milestones so far: - - 3/21/12: We have entries for 2,601 of 3,572 fingerprints (971 - missing; 73% coverage) - - 11/5/12: We have entries for 3,285 of 3,907 fingerpritns (622 - missing; 84% coverage) - - 11/12/12: We have entries for 3,558 of 3,946 fingerprints (388 - missing; 90% coverage). - -o [Zenmap] should actually parse and use script results. See - http://seclists.org/nmap-dev/2010/q1/1108 - - We have an initial prototype, but probably need to redo because it - doesn't present the results in the way we'd like yet due to - problems implementing such a presentation with GTK, etc. - -o Make Zenmap settings get upgraded when the Zenmap executable is - upgraded. The per-user configuration files such as scan_profile.usp - and zenmap.conf are never overwritten once installed by Zenmap, so - changes and fixes to those files don't reach anyone who has - installed Zenmap already. This is most noticeable with changes to - profiles and highlight definitions are notably affected. This fix - may involve hard-coding settings that are not normally configured by - users (like highlighting) or updating the per-user files at startup - (only those parts that haven't been changed by the user). - -o We should offer partial results when a host timeouts. I (Fyodor) - have been against this in the past, but maybe the value is - sufficient to be worth the maintenance headaches. Many users have - asked for this. If we do implement this, we may want to only print - results for the COMPLETED phases (e.g. host discovery, port - scanning, version detection, traceroute, NSE, etc.) Trying to print - partial results of a port scan or NSE or the like might be a pain. - And if we print some results for a host which timeouts, we should - give a very clear warning that the results for that host are - incomplete. As an example, here is someone who hacked Nmap source - code to achieve this: http://seclists.org/pen-test/2010/Mar/108. - o Another benefit would be that it would allow us to clean - up/regularize the host output code. Right now there are I think - three places where a host's final output can be printed. If, - instead, that code just looked at what information was available and - printed that out only, we could potentially isolate it in just one - place. - o This also might let us provide a feature for skipping the rest of - an Nmap phase which is going too slowly (I think that has its own - Nmap TODO item). - -o [Nsock] Some SSL connections that used to work now fail; find out - why. http://seclists.org/nmap-dev/2010/q4/788. Narrowed down to - r19801 in http://seclists.org/nmap-dev/2011/q1/12. - -o [NSE] Consider a system where scripts can tell if any other scripts - depend on them. They could then use that to determine whether they - should bother storing information in the registry. For example, - snmp-interfaces could store the discovered table if another script - (such as a mac address geolocator script) depends on it. - -o [NSE] Consider whether we need script.db for performance reasons at - all or should just read through all the scripts and parse on the fly. - See: [http://seclists.org/nmap-dev/2009/q2/0221.html] - -o A couple minor nsedoc issues (see - http://seclists.org/nmap-dev/2011/q1/1095): - o After the ssh-hostkey portrule was added, nsedoc seems to be - generating a blank "Script types" filed for the script: - http://localhost:8082/nsedoc/scripts/ssh-hostkey.html - o This is happening because "portrule" and "hostrule" appear later in - the script, and NSEDoc thinks it is their definition, and there is - no NSEDoc there. - local ActionsTable = { - -- portrule: retrieve ssh hostkey - portrule = portaction, - -- postrule: look for duplicate hosts (same hostkey) - postrule = postaction - } - o ssh-hostkey and rmi-dumpregistry each have two @output sections, - and NSEDoc is only showing the second one. We should probably just - combine them into one @output section, and maybe make nsedoc give a - warning in this case. Or we could make nsedoc handle multiple - @outputs. - -o Add general regression unit testing system to Nmap - o David has created a system for Ncat which could serve as a - model. - -o Make version detection and NSE timing system more dynamic so that - the concurrency can change based on network conditions/ability. - After all, beefy systems on fast connections should be able to handle - far more parallel connections than slower systems. - o At a minimum, this at least warrants more benchmark testing. - -o We should run at least one SCTP service on scanme. Daniel - Roethlisberger has made available dummy services which support IPv4 - and IPv6 (see http://seclists.org/nmap-dev/2011/q2/450). - Alternatively, we could run some sort of "real" SCTP application(s) - (preferably one which is relatively simple, easy to install, secure, - and supports IPv6). - -o Create new default username list: - http://seclists.org/nmap-dev/2010/q1/798 - o Could be a SoC Ncrack task, though should prove useful for Nmap - too - o We probably want to support several lists. Like an admin/default - list like "root", "admin", "administrator", "web", "user", "test", - and also a general list which we obtain from spidering from - emails, etc. - -o Improve Nsock proxies system - - Add SSL support - - Add proxy authentication - - Switch Ncat to using Nsock proxy system rather than it's own - built-in support. - - Move the code which is shared with ncat to nbase (URL parsing code, - for instance). - - Add socks4a/socks5 support. This requires to figure out how to - enter the nsock proxy code w/o having the target IP address. No huge - technical blocker there though, only design choices to make. - - Nping could potentially use it as well (could be useful for - measuring latency and reliability of a given proxy chain, for - example). - - Add proxy support to connect() scan. This would mean moving - connect scan to nsock. - -o [NCAT] Send one line at a time when --delay is in effect. This is - cumbersome to do until Nsock supports buffered reading. - -o [NCAT] Make the HTTP proxy support the chunked transfer encoding, - then change it to be HTTP/1.1 and support pipelining. - -o [NCAT] Drop privileges once it has started up, bound the ports it - needs to, etc. - -o [NCAT] Work as a SOCKS4a/SOCKSv5 proxy. - -o [NCAT] Resolve names through the proxy when possible. - http://seclists.org/nmap-dev/2012/q2/768 - -o [NSE] Script writing contest (something to think about) - -o We should document an official way to compile/test refguide.xml so - people can more easily test their changes to it. This will probably - involve moving legal-notices.xml into /nmap/docs, among other - things. - o Note that nping has its own /nmap/nping/docs/genmanpage.sh - we - could look at how that could apply to Nmap. - -o Move Zenmap man page from nmap/docs/ to nmap/zenmap/docs to match - the man page location for ncat and ndiff. - o Don't break packaging/build system - o Don't break the system for posting html to web site. - o Consider standardizing names for nping and ncrack man pages as well. - [Fyodor] - -o [NSE] MSRPC - Improve domain support all around -- in particular, - let the user give the domain in the format DOMAIN\username or - username@DOMAIN anywhere that usernames are accepted. Suggested - at http://seclists.org/nmap-dev/2010/q2/389 - -o [NSE] Combine similar MSRPC scripts, especially the "get info" - stuff. See this thread on combining - (http://seclists.org/nmap-dev/2010/q1/1023). This was suggested by - Ron at http://seclists.org/nmap-dev/2010/q2/389. - -o [Zenmap] Investigate getting new OS icon art. See - http://seclists.org/nmap-dev/2010/q1/1090 - -o We should probably enhance scan stats--maybe we can add a full-scan - completion time estimate? Some ideas here: - http://seclists.org/nmap-dev/2010/q1/1007 - -o [NSE] Do some benchmarking of our brute.nse. We should check the - performance with different levels of thread parallelism. Our - initial results show that it isn't helping much for vnc-brute or for - drda-brute (which is currently using the multi-thread feature - directly rather than through brute.nse library). We should figure - out why the threads aren't helping more, and whether there is - something we can do to fix it. It would also be interesting to - compare speed with Ncrack for services we have in common. - -o Start project to make Nmap a Featured Article on Wikipedia. - - See http://seclists.org/nmap-dev/2010/q1/614 - -o Add Nmap web board/forum - - First step is looking at the available software for this. - - Nmap subreddit exists: https://www.reddit.com/r/nmap - -o [Zenmap] Consider a couple ideas from Norris Carden - (http://seclists.org/nmap-dev/2010/q2/228): - - remember last save and/or open location for new saves and/or opens - - default save location option - -o [Nsock] Consider adding server support to Nsock so it can accept - multiple connections and multiplex the SD's, like it does for - clients. This could potentially be used by Ncat and Nping echo - mode. Currently Ncat server doesn't use Nsock at all, while Nping - echo mode basically polls, repeating a loop of 1s in nsock_loop - followed by a nonblocking accept(). Then Nping gives the SD's to - Nsock to manage. - -o Consider implementing both global and per-host congestion control in - the IPv6 OS detection engine. Currently it handles congestion globally - (one CWND and SSTHRESH shared by all hosts). This works fine but it - may not be the most efficient approach: if the congestion is not - in our network segment but in a target's and we are os-scanning - hosts in different networks, then all hosts get "penalized" because - there is congestion in another network, not in theirs. - -o [Nsock] Consider implementing a nsock_pcap_close() function or making - nsp_delete() call pcap_close() when pcap IODs are used. Currently valgrind - warns about a socket descriptor left opened (at least in Nping). - ==10526== at 0x62F77A7: socket (syscall-template.S:82) - ==10526== by 0x4E348A5: ??? (in /usr/lib/libpcap.so.1.0.0) - ==10526== by 0x4E36819: pcap_activate (in /usr/lib/libpcap.so.1.0.0) - ==10526== by 0x4E375FC: pcap_open_live (in /usr/lib/libpcap.so.1.0.0) - ==10526== by 0x4311A9: nsock_pcap_open (nsock_pcap.c:64) - ==10526== by 0x428078: ProbeMode::start() (ProbeMode.cc:329) - -o Consider rethinking Nmap's -s* syntax for specifing scan types - o Current problems with this -s syntax: - o We already use like 20 of the 26 letters, so we end up with - things like SCTP scan using -sY - o Can make Nmap command lines hard to read, particularly given - that we often need to improvise to find a letter which isn't - taken. - o Problematic for scan types -sI and -b which require arguments - o Inconsistencies. For example, -sC and -sV do script scan and - version detection, respectively, and yet for OS detection we use - -O. Also, control flow (-sP, -sL) is used with -s, which further - overloads the options. - o Possible solution: - o We are enabling -Pn and -sn as preferred notations for -PN and - -sP which mean "no ping" and "no port scan". Those match the - already existing -n for "no DNS". The problem with -sP is that it - implies "ping only", when what it really should mean is "disable - port scan" because you may want to do NSE, OS detection, - traceroute, etc. still. - o We might want to just give them normal option strings, so you - could do --maimon instead of -sM, for example. For extremely - common options such as SYN scan, UDP scan, version detection, we - could perhaps find good single letter options as an alias to the - longer one. - o Another idea is to use something like --scantype syn,udp,sctp, - which is a lot longer for single-type scans, but shorter when - you're combining mulitiple ones. Doesn't allow for individual - scan arguments easily. I (Fyodor) think I prefer the idea above - of just givem them top level arguments. - o If we keep -s*, we could just give it one defined function, such - as selecting port scan type, or control flow. - o Obviously this will take some discussion/brainstorming on nmap-dev. - -o Do -p- Internet UDP scans. - -o Scanning through proxies - o Nmap should be able to scan through proxy servers, particularly now - that we have an NSE script for detectiong open proxies and now that - Ncat can act as proxy client or server. - o Requirements: - o Would be nice to be able to chain through multiple proxy servers of - different types. - o Would be nice to be able to spread the load amongst multiple - proxies. - o Should support port scanning, version detection, and NSE. In - other words, nsock should support proxies. - o Support IPv4 and v6 - o Need to figure out how to get good performance. Pool of - connections to proxy or proxies for concurrency? HTTP pipelining? - o Support the different varieties of proxies: socks4, socks4a, - socks5, HTTP GET (if possible), HTTP CONNECT. Note that GET - proxies present some challenges since the error messages may not - be standard, etc. - o Maybe auto-detect the proxy type so that Nmap can try the most - efficient scanning method first? - o I've been asked to support basic, ntlm, and digest authentication - if possible. - o Implementation ideas: - o There is a patch by Zoltan Panczel (http://nmap-dev.fw.hu) and it - has been improved by Jacob Appelbaum in nmap-exp/ioerror/ . This - patch doesn't handle things like parallelization, but it may be a - good proof of concept. - o This might not be appropriate for ultra_scan ... perhaps would be - better to write a general scanning engine for abusing - applications for port scanning purposes. This could handle - scanning through proxies and the existing FTP bounce scan would - also be ported to this engine (or, frankly, we could probably get - away with removing FTP bounce). rembrandt at jpberlin.de tells me - that you can also do this with the "forwarding" commands on IMAP - servers. Whoever does this should probably start by reading the - code for the main port scanning engine (ultra_scan()) and also - the version detection code (service_scan()). And the version - detection paper at https://nmap.org/book/vscan.html. If you - understand all that, you may be ready for this project :). This - is important, because it is easy to do poorly. The tough part is - high performance and clean code which is general enough that all - these different applications can be scanned through using the - same basic engine. You should run your ideas by nmap-dev in as - much detail as possible before starting. - o David: I'm starting to think about building proxy support into - Nsock and then implementing -sT with Nsock instead of ultra_scan. - -o [Web] Consider adding training/introduction videos to the Nmap site - o Would be great to have a (5 minute or less) promotional video - introduction to each tool (Nmap, Zenmap, Ncat, Ndiff) on its web - page. - o They need to be good to be useful--the sort of the quality you see - in Laura Chappell's Wireshark videos or James Messer's Nmap videos - or Irongeek's videos (http://www.irongeek.com). - o Besides the promotional videos, users would probably enjoy more - in-depth video instructions (e.g. covering the Nmap Network - Scanning topics). - o Here's an example product page with lots of videos (we may not go - that far): http://www.splunk.com/product - -o The Zenmap translation system - (https://nmap.org/book/zenmap-lang.html) has been pretty successful - so far. We should consider doing the same for Nmap. After all, we - already have the reference guide in 16 languages at - https://nmap.org/docs.html. We should definitely try to use the same - translation methods for Zenmap as we do for Nmap. In fact, maybe we - can create a combined PO file Nmap, Zenmap, Ncat, and Ndiff so that - they can all be translated and maintained together. Something to - consider: calling setlocale can change the behavior of functions like - isalpha. Locale-dependent functions need to be checked for security - risks. - -o [NSE] Consider whether we should include some sort of NSE debugger. Or we - could include something simpler. For example, Nmap now provides a - traceback (with sufficient debugging/verbosity) when a script ends - in error. For some inspiration/ideas, look at Diman's NSE - debugger (http://seclists.org/nmap-dev/2008/q1/0228.html). - -o [NSE] Support routing http requests through proxies. - -o [NSE] Would be great if NSE scripts could be made to NOT - run as root if they don't have to. - -o [NSE] Security Review - o Consider what, if any, vulnerabilities or security risks NSE has - with respect to buffer overflows, format string bugs, any other - maliciously formatted responses from target systems, etc. Maybe - address the known risk of malicious scripts too. - o Consider that NSE runs scripts as root - -o More security auditing of Nmap code (it never hurts to do more proactive - security auditing). - -o Figure out and document (in at least the Ncat user's guide) the best - way to use Ncat for chaining through proxies. One option is this - sort of thing: - ncat -l localhost 1234 --sh-exec "ncat --proxy A.A.A.A B.B.B.B" - ncat --proxy localhost:1234 C.C.C.C - If you had two proxies A.A.A.A and B.B.B.B, connecting to C.C.C.C. - With another listener/--sh-exec pair for each additional proxy. - But perhaps we can make it easier by adding it to the syntax. - -o Look into whether we should loosen/change the global congestion - control system to address possible cases of one target host with many - dropped packets slowing down the whole group. See - http://seclists.org/nmap-dev/2008/q1/0096.html . - * Related possibility: Fix --nogcc to gracefully handle ping scans. - Right now it seems to go WAY TOO FAST (e.g. several thousand - packets per second on my DSL line). - * [12/22/09] David says: It still is in one case that I've - documented on my wiki. I had an idea to fix it, but on testing it - it didn't work. The idea was to treat the global congestion limit - differently. Instead of dropping it down to the minimum level on a - drop as is done currently, I thought about only dropping it by the - amount that the individual host limit drops. For example, if a - host had a drop and its limit fell from 25 to 1, then the global - limit would change (if it was at 100 to begin with) to 76, not all - the way down to 2 or whatever it is. The idea being that the - global limit is most important at the beginning of a scan, when - there's no information to set host limits, and every host wants to - send all its first probes at once. See - http://www.bamsoftware.com/wiki/Nmap/PerformanceNotesArchive2#global-cc. I - am convinced, though, that some sort of global control is - necessary. There's a reason that a web browser limits the number - of connections it will make, and doesn't try to download every - image file at once and count on the fairness of TCP to sort it - out. - -o libnmap organization for UNIX and Windows - o Then change Nmap and Zenmap to simply call this library - o It is interesting to look at: http://www.gnupg.org/gpgme.html - -o Deal with UDP retransmission for version detection (I think I - should just do a second run of all probes for UDP if it fails to - match anything). The advantage there is that no retransmissions are - neccessary if the service is found. Then again, per-probe - retransmission would let us redo the most likely probes (the one(s) - that match the port number) quickly. Lost packets should probably - affect ideal_parallelism. - -o Make RPM relocatable (requires somehow avoiding storing paths in the - binary) - - That may be easier now that David has made some big improvements - in detecting where the binary is cross-platform and then looking for - data files based on that location. - -o Nmaprc-related - Create a system to store Nmap defaults/preferences - in an nmaprc file. - o nmaprc should be in ~/.nmap on UNIX - o On Windows, we may need a registry key to find the .nmaprc - o Perhaps Lua could be used as the format? - o .nmaprc for keeping defaults, etc. - o Nmaprc infrastructure, hook to new timing variables - o Nmaprc man page - o Default timing mode - o Default NSE arguments, such as user agent - o Maybe Default source IP (-S) argument - o should be a way to specify your own .nmaprc - o Maybe lets you add a directory and template for saving all - scans. - o Maybe let you define "scan profiles" like is done with Zenmap. - There would then be a command-line option to select the profile used. - -o Get new Zenmap logo - o consider putting back on top-right of command constructor wizard - (there used to be umit logo there). - o Maybe that can be done after the release by soliciting ideas. - -o Create or collect some great ./configure ascii art. - -o Look at all the pcap functions, there are some like - pcap_findalldevs() which could be quite useful. There are mails to - the Nmap list relating to suggested improvements -- - http://seclists.org/lists/nmap-dev/2004/Apr-Jun/0024.html . - Actually I do indirectly use that for Windows. I wonder if they work - for UNIX? - -o perhaps each 'match' line in nmap-service-probes should have a - maximum lines, bytes, and/or time by which a response should be - available. Once that much time (or many bytes or lines) have passed, - that match can be considered 'failed' and ignored in subsequent runs. - Once all matches are considered failed, that probe is done. This - could be a useful optimization and is arguably better than the less - granular 'totalwaitms'. Or I could just have a simple function that - looks at whether a given regex could possibly match something - starting with the received data (not too hard since almost all of - the current regexes are anchored). But before doing this, I should - look long and hard at how many of the probes have every match - capable of doing this. In particular, many of the softmatch lines - don't offer many chars anchored at the front. - -o Separate nbase into its own Windows library in the same way as Andy did - with iphlpapi . - -o Nmap / Nmap-hackers FAQ - -o random tip database - diff --git a/todo/nping.txt b/todo/nping.txt deleted file mode 100644 index c1130cf30..000000000 --- a/todo/nping.txt +++ /dev/null @@ -1,799 +0,0 @@ -/***************************************************************************** - * * - * o * - * o * - * o * - * o o * - * o o * - * o o * - * o o o * - * o o o * - * 888b 888 o o o * - * 8888b 888 o o o * - * 888Y88 888 o o o * - * 888Y88b 888 o * - * 888 Y88b888 o * - * 888 Y88888 * - * 888 Y8888 * - * 888 Y888 * - * * - * --[NPING TO-DO LIST]-- * - * * - *****************************************************************************/ - - This file contains Nping's to-do list. Items are listed in order of priority - (high priority items are listed first). Feel free to work on any of the items - on the list. However, if you'd like to work on something that is not trivial - to implement you may want to send a message to the nmap-dev list before you - start so other developers can see what you are planning to do. Make sure you - explain exactly what you are trying to fix/implement and how you are planning - to do it. It's always better to discuss bugfixes and new feature additions in - advance because they may actually have bigger implications than you think and - you may not get your patch accepted. - - Please keep in mind that contributed code must: - * Be written in C++. - * Include comments so anyone can understand immediately what it does. - * Work on Linux, Mac OS and MS Windows. It's OK if you have not tested - the code in all those platforms, but at least keep portability in mind when - you write it and include a list of systems you've tested it on along with - your patch. - - Questions, comments and patches should be sent to the Nmap development - mailing list (nmap-dev). To suscribe: - - - -/***************************************************************************** - * Things that have NOT been done yet * - *****************************************************************************/ - -* Improve IPv6 support. Currently it doesn't work well. The situation should be - analyzed in detail because right now Nping has code to send packets at raw - transport level (letting the OS craft the IPv6 header), and at raw ethernet - level. None of them seems to work well, though. - -* Investigate an IPv6-related core dump reported by Vasiliy Kulikov. - More info: http://seclists.org/nmap-dev/2011/q3/567 - -* Consider using Nmap's proto-dependant payloads for UDP packets. According - to David's tests, better results are obtained when sending UDP probes with a - payload specific to the protocol. - -* Consider adding the possibility to see the RTT in the RECV line. Something - similar to the way the traditional ping tool prints the RTT (time=XXX ms) - - $ ping nmap.org - PING nmap.org (173.255.243.189) 56(84) bytes of data. - 64 bytes from nmap.org (173.255.243.189): icmp_req=1 ttl=48 time=169 ms - 64 bytes from nmap.org (173.255.243.189): icmp_req=2 ttl=48 time=177 ms - 64 bytes from nmap.org (173.255.243.189): icmp_req=3 ttl=48 time=179 ms - ^C - --- nmap.org ping statistics --- - 3 packets transmitted, 3 received, 0% packet loss, time 2000ms - rtt min/avg/max/mdev = 169.097/175.137/179.152/4.347 ms - - - This was requested by Jacek Wielemborek. More info: - http://seclists.org/nmap-dev/2013/q3/533 - -* Currently, Nping determines the maximum number of open descriptors - (in TCP connect and UDP unprivileged modes), from the value returned - by libnetutil::get_max_open_descriptors(). However, it is often the - case that such function returns a value higher than FD_SETSIZE, which - is the maximum number of descriptors that select(2) can handle. - Currently Nsock uses select(2) so we have to limit the number of - descriptor to FD_SETSIZE, and not to the value returned bu - get_max_open_descriptors(). However, Henri Doreau is working on a new - nsock-engines branch which will provide Nsock engines based on - better I/O syscalls like poll() and epoll(). I've asked Henri if he - could implement a function in Nsock that provides the maximum number - of descriptors that can be handled at the same time, based on the - nsock engine being used. So, if that function gets implemented and - his nsock-engines branch merged into trunk, we should consider - updating Nping's code to use it. - More info here: - http://seclists.org/nmap-dev/2011/q4/550 - -* A few ideas for the Echo protocol: - - Add an authenticated NEP_BYE message, so session termination is explicit - and both ends can determine if the session was ended because the other end - requested it or if it was due to some error at the network or transport - layer. Suggested by David. - - - Add examples for encryption and hmac to the RFC. This would help in - debugging implementations. Suggested by Toni Ruottu. - - - RFC. Improve description of how the IVs work. Suggested by Toni Ruottu. - - - RFC. Improve description of encryptionless sessions. Suggested by Toni - Ruottu. - - - Currently, the echo server zeroes any application layer data before - transmission in a NEP_ECHO message. This minimizes the impact of - errors in the server's packet matching engine or malicious attacks that - attempt to trick the server into echoing packets that do not belong to - a particular user. This works well but in the future, if one day we - create a NEPv2 specification, we may want to consider extending NEP_ECHO - packets to allow stripped-packet transport. This is, to allow echo servers - to remove application layer data before transmission, and include - additional information in the NEP_ECHO message so clients can determine - that the payload part was stripped and how long was it. - - - Consider making the echo server bind to all IPv4 AND IPv6 interfaces. - - - Add a description of the security implications of running a public echo - server (failures in the packet matching algorithm, etc), to either the - RFC or the man page. Suggested by Toni Ruottu. - - - Test the new --safe-payloads option with a packet fuzzer to make sure - the packet parser behaves correctly. - -* When running Nping echo client with the --no-capture parameter, the last - packet's CAPT line is not displayed. - - nping --ec public echo.nmap.org -p90 --tcp --count 1 --no-capture - - luis@Aberdeen:~$ sudo nping --ec public echo.nmap.org -p90-92 --tcp --count 1 --no-capture - - Starting Nping 0.5.52.IPv6.Beta2 ( https://nmap.org/nping ) at 2011-07-05 12:53 CEST - SENT (7.3302s) TCP 163.117.203.253:18554 > 74.207.244.221:90 S ttl=64 - CAPT (7.4625s) TCP 163.117.203.253:18554 > 74.207.244.221:90 S ttl=54 - SENT (8.3309s) TCP 163.117.203.253:18554 > 74.207.244.221:91 S ttl=64 - CAPT (8.4429s) TCP 163.117.203.253:18554 > 74.207.244.221:91 S ttl=54 - SENT (9.3310s) TCP 163.117.203.253:18554 > 74.207.244.221:92 S ttl=64 - - Max rtt: N/A | Min rtt: N/A | Avg rtt: N/A - Raw packets sent: 3 (120B) | Rcvd: 0 (0B) | Lost: 3 (100.00%)| Echoed: 2 (80B) - Tx time: 2.00181s | Tx bytes/s: 59.95 | Tx pkts/s: 1.50 - Rx time: 2.00193s | Rx bytes/s: 0.00 | Rx pkts/s: 0.00 - Nping done: 1 IP address pinged in 9.33 seconds - -* Sometimes Nping displays a couple of error messages (related to cleanup of - Nsock events), even though everything went fine. - - luis@Aberdeen:~$ sudo nping --ec public echo.nmap.org -p90 --tcp --count 1 - - Starting Nping 0.5.52.IPv6.Beta2 ( https://nmap.org/nping ) at 2011-07-05 12:51 CEST - SENT (1.8965s) TCP 163.117.203.253:64288 > 74.207.244.221:90 S ttl=64 - CAPT (2.0293s) TCP 163.117.203.253:64288 > 74.207.244.221:90 S ttl=54 - RCVD (2.1233s) TCP 74.207.244.221:90 > 163.117.203.253:64288 RA ttl=51 - nping_event_handler(): READ-PCAP killed: Resource temporarily unavailable - nping_event_handler(): TIMER killed: Resource temporarily unavailable - - Max rtt: 226.762ms | Min rtt: 226.762ms | Avg rtt: 226.762ms - Raw packets sent: 1 (40B) | Rcvd: 1 (40B) | Lost: 0 (0.00%)| Echoed: 1 (40B) - Tx time: 0.00136s | Tx bytes/s: 29411.76 | Tx pkts/s: 735.29 - Rx time: 1.00082s | Rx bytes/s: 39.97 | Rx pkts/s: 1.00 - Nping done: 1 IP address pinged in 2.93 seconds - -* Investigate about warning on old version of gcc like g++ 4.1.2 20080704 - (Red Hat 4.1.2-48). No warnings are shown on newer version but it would be - nice to get rid of them if possible. There are some of them: - - ARPHeader.h:169: warning: ‘class ARPHeader’ has virtual functions but - non-virtual destructor - RawData.h:99: warning: ‘class RawData’ has virtual functions but - non-virtual destructor - -* Decide more on rDNS - - Do we want to rDNS resolve all target IPs? If so, where should we - show the name? At the final report (even when just one host - scanned, which omits that line now)? In the individual packet - trace lines? When a CNAME (or a name which forward resolves but - does the IP doesn't reverse resolve) is specified on the command - line, should we use that version, or the official rDNS, if any? - - Some more discussion on this topic on nmap-dev may be warranted. - -* Improve output for negative verbosity levels. Currently, one can't - even tell how many hosts replied, just how many responses were - received, which could be all from the same host. If there is only - one target, then the current behavior is fine. However, when pinging - more targets, we should be able to provide a better output; at least - how many hosts were alive. This was suggested by Dan Farmer. - -* Consider adding more examples of setting fields/payloads to the man - page. This was suggested by Dan Farmer. - -* Consider adding support for XML output. - -* From: David Lam , "Some general questions about - Nping/Ncat" - - In TCP traceroute mode, would it be possible to ask Nping to - stop once it gets an SYN-ACK response back from the destination host rather - than continuously hitting the host until the max TTL? - -* Make broadcast ping work. Currently the following command does not - show any captured packets: - nping 192.168.0.255 --dest-mac ff:ff:ff:ff:ff:ff -c 1 - The cause is probably the BPF filter, which only allows replies from - 192.168.0.255. - Also, look into official multicast addresses like 224.0.0.1. Can we - received replies to that probe? - - -* Do some performance testing. - Fyodor: - <> - -* Stats for ARP packets. - -* Do more testing on Mac - -* Support pre defined probe rates: --fast, --faster, --flood, --slow, - --slower, --paranoid... - -* Think about --establish feature, which uses raw packets to establish - a connection and can then send data on the connected stream (Luis - already has a proof-of-concept implementation). - -* Make privileged and unprivileged TCP/UDP mode specification consistent. - -> - User is unprivileged and did not supply mode: --> Use TCP-Connect -> - User is unprivileged and supplied --tcp --> Use TCP-Connect -> - User is unprivileged and supplied --upd --> User UDP unprivileged -> - User is root and did not supply mode --> Use ICMP Echo -> - User is root and supplied --tcp --> Use raw sockets TCP -> - User is root and supplied --udp --> User raw sockets UDP -> - User is root and wants to use TCP-Connect --> User needs to either -> pass --tcp-connect or --unprivileged -> - User is root and want unprivileged UDP --> User needs to pass -> --unprivileged or --udp-XXXXX (any suggestions?. --udp-sendto() may not -> be the best idea because when we use raw sockets we also use sendto() to -> transmit the data). - -* Support reverse DNS resolution in --traceroute - -* Implement TCP options - -* Implement hping-like ability to change the port/ttl using the keyboard - during a scan. - -* Disable ARP resolution when --source-mac is specified. - -* Implement --data-file option. What should we do if file is big? Read the - first X bytes? Send consecutive chunks? - -* Implement ICMP address mask - -* Implement entire ICMP Traceroute message opts. - -* Research on default IP Identification value. Kernel does not seem to like - value 0 because when set to zero, kernel changes it to some other value. When - we set it to something !=0, the kernel leaves our value untouched. - -* At some point in the future, implement weird ICMP Types. I think this would - let us make a difference to the rest of pings and packet creation tools - because anyone wanting to send weirds packes would have to download our - Nping ;-) - ( http://www.iana.org/assignments/icmp-parameters ) - 6 Alternate Host Address [JBP] - 31 Datagram Conversion Error [RFC1475] - 32 Mobile Host Redirect [David Johnson] - 33 IPv6 Where-Are-You [Bill Simpson] - 34 IPv6 I-Am-Here [Bill Simpson] - 35 Mobile Registration Request [Bill Simpson] - 36 Mobile Registration Reply [Bill Simpson] - 39 SKIP [Markson] - 40 Photuris [RFC2521] - -* Implement checks in function that handles received packets: - Fyodor: - <> - -* Implement "-iL inputfilename (Input from list) " and the case where "-" is - supplied and target specs need to be read from stdin. - -* Consider adding option to allow sending NO packets but act as a - simple sniffer. Users could use --bpf-filter to specify a - tcpdump-like filter and get every receive packet printed to - stdout. Maybe with "-c 0"? "-c none"? We need to have some flag in - NpingOps so we don't terminate Nping but wait undefinitely. - -* At some point we should support nmap-like MAC specification. - -* When implementing IPv6, check MAX_TCP_PAYLOAD_LEN constant and method - TCPHeader::setSum(). Because with IPv6 the max payload length should be 20 - bytes less than with the IPv4 header. - -* When using payloads, take into account that the IP and TCP headers may - contain options and therefore, the maximum payload len should be - 65535 - 20(ip header) - 40 (ip options) -20(tcp header) -20(tcp options); - -* Make sure randomnly generated checksums in IPv6-TCP/UDP are in fact invalid - and don't match the correct checksum. - -* Fyodor: - <> - -* ARP mode does not support payload specification. However, users may - want to do things like appending null bytes at the end of an ARP - packet to test some device behaviour, etc. Adding support for - payload to this mode is really trivial, would make the payload spec - more consistent with the rest of the modes, and may be a nice to have - feature. - -* [EM] For CAPT packets, decide if we want to print the full info or - just the fields that have changed in transit (or both). Note that - printing differences would be complicated by the fact that nping - doesn't currently associate captured packets with the original send. - -* Decide if we want to allow things like "1074628148" or "0x400d8634" to - be treated as valid IP addresses. - -* Check out if --ip-options "RTUS 1.1.1.1 2.2.2.2" makes sense. It now - fails. - -* It may be nice to let users set the IP header lenght field. Maybe they - want to stress tcp/stacks with this. - -* Investigate on ICMP preference levels. It's not clear whether there is - a standard encoding or not. The logic that parses this in Nping needs - to be reviewed. - -* Split up libnetutil.cc into different source files. - -* Investigate on nping's version of devname2ipaddr. Think about side - effects on using that in Nmap. - -* Consider adding multi-packet support. - o Example: tell nping to send 4 tcp packets, 5 icmp packets and 3 udp packets - -* Consider adding RFC-style output for send/recv packets. - -* Consider adding more detailed stats for the Echo Mode. - -* [EM] Handle DLT types. Currently the server always sets the null DLT value - that indicates that no data link header is included. - -/***************************************************************************** - * Things that have been solved already * - *****************************************************************************/ - -[DONE] Add default target port for TCP-Connect and TCP modes :: Port 80 - -[DONE] Add default target port for UDP mode :: Port 40125 - -[DONE] Add default UDP Source port: 53 - JUSTIFICATION: From David's EffectivenessOfPingProbes - http://www.bamsoftware.com/wiki/Nmap/EffectivenessOfPingProbes - "The best individual UDP probes are still those to a random high port, - with a source port of 53 and a non-empty payload. Even without the source - port and payload, the ports 40125 and 40126 that I picked out of the air - are better choices than the current default of 31338, finding around 400 - additional hosts." - -[DONE] Change resolution for the inter-ping delay. (Fyodor: btw, usleep() will - probably do the trick for you as it let's you sleep with microsecond - precision) - -[DONE] Use int send_ip_packet(int sd, struct eth_nfo *eth, u8 *packet, unsigned int - packetlen) instead of ip_open(); - -[DONE] Add protocol to BPF filterstring because It is possible that when in TCP mode - a UDP packet destined to the TCP source, arrives to the net iface and gets - printed. - -[DONE] Implement multiple port specification. - -[DONE] Implement ICMP router advertisement entries - -[DONE] Default probe mode: ICMP echo - -[DONE] Test ICMPv4Header::addRouterAdEntry() and check entries are being added - correctly. - -[DONE] Determine source IP address automatically - -[DONE] Determine network interface to be used for packet capture automatically - -[DONE] Add support for cached DNS requests - -[DONE] Start user documentation (mainly man page) - -[DONE] Change output to include timing information - -[DONE] Implement controls in payload options parsing to prevent specifying lengths - that cannot be carried by a single TCP/UDP packet. - -[DONE] Start implementing unprivileged UDP pings. - -[DONE] When sending ICMP packets, checksum is not being computed correcly if - --data-length, and options like that, are specified. - -[DONE] Find a bug that under some circumstances produces a segfault. It is probably - related to the way option -e is being handled. - -[DONE] Fix a bug in option "-e iface" that results on IP 2.0.0.0 being used as a - source address. - -[DONE] Update --help display to include new ICMP flags. Check also commandline syntax - docs. - -[DONE] Use nsock approach instead of threads. - -[DONE] Finish ARP/RARP support. - -[DONE] Change doc for option --count. We don't stop after N probes, we stop after - N rounds. - -[DONE] Ask Fyodor what tool is used to convert from nmap-man.xml to nmap.1 - -[DONE] Check all outPrint()s and outError()s to ensure they specify the correct - verbosity/debug level. - -[DONE] Document format specified in ArgParser::atoICMPType(). - -[DONE] Document format specified in ArgParser::atoICMPCode(). - -[DONE] Finish implementing unprivileged UDP pings. - -[DONE] Finish Ethernet frame creation. - -[DONE] Find a way to convert the nping.xml into man page. - -[DONE] Check what happens if payload is specified and we are not sending TCP/UDP - but ICMP or other proto packets. [Sometimes it may not make sense to include - payloads (e.g. ARP) but we still allow it just in case users want to play - around]. - -[DONE] Ask Fyodor whether we want to display elapsed time (like nmap) or we prefer to - display rtt time as other ping utilities do. [This is probably fine for now] - -[DONE] Fix the warnings produced by Fyodor's gcc. - +---------------+ - NpingTargets.cc: In member function ‘int NpingTargets::processSpecs()’: - NpingTargets.cc:315: warning: comparison between signed and unsigned integer expressions - NpingTargets.cc: In member function ‘NpingTarget* NpingTargets::getNextTarget()’: - NpingTargets.cc:333: warning: comparison between signed and unsigned integer expressions - +---------------+ - In file included from /usr/include/string.h:640, - from nbase/nbase.h:158, - from nping.h:107, - from utils.cc:95: - In function ‘void* memset(void*, int, size_t)’, - inlined from ‘int getNetworkInterfaceName(sockaddr_storage*, char*)’ at utils.cc:689: - /usr/include/bits/string3.h:85: warning: call to void* __builtin___memset_chk(void*, int, long unsigned int, long unsigned int) will always overflow destination buffer - +---------------+ - - -[DONE] Redesign verbosity levels: - * Put verbosity levels 2 into level 1 - * Use level 2 for error. - * Use level 3 to print everything but not sent/rcv packets. - * Level 4 the usual - -[DONE] Add stats at the end of nping execution. - -[DONE] Add options to disable viewing of sent packets. - -[DONE] Add option to to disable packet capture. - -[DONE] Add a section to the man page explaining how we iterate over targets, - ports, etc. - -[DONE] Beta-testing email to the list. - -[DONE] Change default round count to 5. - -[DONE] Fix a segfault detected by Fyodor in trg=o.targets.findTarget(...). - -[DONE] Send an email to the list telling about the nping.exe file. - -[DONE] Support CTRL-C statistics. - -[DONE] Change "solution" file in mswin32/nmap.sln to nping.sln - -[DONE] In man page and -h: move Ethernet section so it appears after network - layer info. - -[DONE] Make rx time more accurate taking into account that we wait for a bit after - the last probe is sent. - -[DONE] Fix bug: add ICMP dest unreachable, etc to the BPF filter so we can get - icmp error messages when TTLs expire, etc. - -[DONE] Disable all ethernet related code when sendEth is false. - -[DONE] Finish porting Nping to Windows. - -[DONE] Find an OS X box to test Nping. - -[DONE] Reorganize verbosity levels (again ;-) [-3, +3]. - -[DONE] Finish documentation for options --source-mac and --dest-mac - -[DONE] Make sure --ether-type supports specifying types in hex. - -[DONE] Implement verbosity level 3: in this level, sent and recv packets are - hexdumped to stdout. - -[DONE] Write and check in nping/index.html web site - - Include SVN checkout/install instructions - - include tarballs when available - -[DONE] Create Windows installer (maybe can copy a lot of stuff from what - Ithilgore has done with Ncrack) - -[DONE] Create Nping release tarball for UNIX systems - -[DONE] Release Nping 0.1BETA2 - -[DONE] Man page should say Nping is currently in Alpha stage. - -[DONE] Support -vvv, -qqq and -ddd syntax. [Requested by Dirk Loss] - -[DONE] Create Mac OS X installer (also can probably copy a lot of stuff - from what Ithilgore has done with Ncrack. David can usually help - with installer building). - -[DONE] Move nping to /nping in SVN rather than being in nmap-exp - -[DONE] Set up automatic conversion from nping XML man page to HTML for - https://nmap.org/nping/man.html [Fyodor working on this] - -[DONE] Include signature files in new releases. [Requested by Henri Salo] -[DONE] It would be nice to have Bzip2 packages. [Requested by Henri Salo] - (These last two don't make sense anymore as Nping is now distributed - with Nmap). - -[DONE] Do small fix in nmap's send_ip_packet_sd() - - res = Sendto("send_ip_packet", sd, packet, packetlen, 0, - + res = Sendto("send_ip_packet_sd", sd, packet, packetlen, 0, - -[DONE] Correct BPF filter specs, to make the condition about the source - address apply everywhere. - -[DONE] Fix possible bug in BPF filter specification. More details in - http://seclists.org/nmap-dev/2010/q2/252 - -[DONE] Work on nping&nmap code merge. - -[DONE] For options that take numbers we need to allow users to specify them - also in hex with the format 0xNNNN... - -[DONE] Replace this pattern: - if ( isNumber_u32(optarg) ){ - u32 aux32 = strtoul( optarg, NULL, 10); - ... - } - with a function that checks for syntax and returns the value (i.e., a wrapper - around strtoul). There is nowhere that isNumber_u* is called without it being - immediately followed by a strtoul, outside of utils.cc. - -[DONE] Bug in --icmp-advert-entry. Specified IPs are being set in host byte - order instead if in network byte order. - -[DONE] Investigate why ARP replies are not being received. Wireshark shows - replies but they don't get captured by Nping. The bpf filter looks - ok: "arp and arp[6]==0x00 and arp[7]==0x02" - -[DONE] Investigate into this: - sudo nping --icmp scanme.nmap.org -vvv -d1 --icmp-type ra --icmp-advert-entry 256.257.258.259,222 - Invalid Router Advertising Entry specification: Unable to resolve 6628128 - Apparently the call to outFatal() is specifying %d instead of %s, but - that's not being detected properly by the compiler, because we don't - get a warning. We have to do something like this: - void fatal(const char *fmt, ...) - __attribute__ ((noreturn)) - __attribute__ ((format (printf, 1, 2))); - TODO: Look at the documentation to see what the numbers mean. - Probably one of the is the index of the format argument, and the - other is where the varargs start. - -[DONE] Fix division by zero exception: - sudo nping --icmp scanme.nmap.org -vvv -d1 --icmp-type echo --rate 0 - ./test_nping.sh: line 83: 11690 Floating point exception"$@" - -[DONE] Fix little problem in TIMING_5. We need to detect the bogus time - before we actually pass the value to NpingOps. Nping is giving an - error but the bogus input is getting to far. - -[DONE] Document that badsum-ip may not always work because the kernel may - correct the sum. - -[DONE] Change overloaded functions in libnetutil that were refactored to - make them compile in C. Go back to the overloaded version if possible. - -[DONE] Move grab_next_host_spec() and pals to netutil. - -[DONE] Control the case when user passes "--mtu 0". An assertion fails but - Nping should print a nicer message. - -[DONE] Improve error message for --mtu. We should probably allow mtu's bigger - than 2^16 but take that as a "dont fragment" request. Also, make - "rand" produce only valid MTUs (multiple of 8, etc). - -[DONE] When passing "--tcp-flags 0x100" the error is not very accurate. - This is because parser_u8() fails and then Nping tries to resolve the - value letter by letter. Maybe we can parse_u32() it, and then check - if n<255 and print a better error message. - -[DONE] Document what happens with the IP header length when user wants to - add uneven bytes of IP options. We are truncating the result, because - the header length is expressed in 32 bit words. - -[DONE] Check if there is any problem with -e "". Maybe we shouldn't let users - supply a NULL name, but make them use the "any" specifier. Add doc - about this and update the test description (MISC_12). - -[DONE] Update documentation for option --delay, including that now, time - specification as float numbers is supported (eg: --delay 0.1 meaning 100ms) - -[DONE] Change info about TODO file in https://nmap.org/nping web page. - - If you wish to contribute code to Nping there is a TO-DO list you can have - - a look at (file "TODO" in the source package). - + If you wish to contribute code to Nping there is a TO-DO list you can have - + a look at (file "todo/nping.txt" in nmap's source package). - -[DONE] Make sure randomnly generated checksums are in fact invalid and don't match - the correct checksum. There is a 1/65535 chance of this happening. - -[DONE] After merging nmap-dedup, change send_frag_ip_packet() to take "u32 mtu" - and fix the printf below to use "%u" instead of "%i". - -[DONE] [EM] Update EchoProtoRFC.txt and any of the other design files as - appropriate and send to nmap-dev for comments - -[DONE] [EM] Pick a default port number - -[DONE] [EM] Make a mockup of the desired standard output in a regular echo mode - execution, like nping -c 2 --tcp --flags SYN -p 80 scanme.nmap.org (let's - assume there are some differences found, like a NAT is in place) - o A key aspect of this task is determining what diffs are going - to look like. - -[DONE] [EM] Things to decide on: - o Decide on packet specifiers that can be passed to the server so it - can recognize packets sent by the client even if a number of headers - have changed and pass them back. (see Fyodor/Luis IM discussion logs - from 6/28/10). - -[DONE] [EM] Improve client error handling. Currently it doesn't behave well when - the server crashes. - -[DONE] [EM] Make the client timeout if the server does not send data during - handshake. Currently the client waits forever. - -[DONE] [EM] Make the server detect when a client disconnects and delete its context - data. - -[DONE] [EM] Get rid of some messages that are currently displayed in the client. - Print them only if debugging level is high enough. - -[DONE] [EM] Make sure -h help screen includes info about the echo mode. - -[DONE] [EM] Add echo mode to the man page. - -[DONE] [EM] Add received echoed packet to the final statistics. - -[DONE] [EM] Multi-client support - -[DONE] [EM] Delay RECV message printing so the CAPT messages are shown in order. - -[DONE] [EM] Use NEP_QUIT only if necessary, just close connection if possible. - -[DONE] [EM] Implement crypto - -[DONE] [EM] Consider whether the CAPT line should (or should have an - option to) display the time based on capture time from the server. - Obviously this can be problematic because not all machines run - ntpd. One option is to just make it an option so that people should - only use it if both the client and server are running ntpd. Luis is - adding a precision timestamp to NEP_ECHO packets so we could easily - add it in the future. Another approach would be to do NTP-style - handshaking to compute time offsets between the two machines during - the echo side-channel handshaking. Then the client could remember - how far off it is. A third approach is to guess about the CAPT time - that it was 1/2 the time between packet send and when we received - the NEP_ECHO back notifying us of receipt. - NOTE: We finally decided to take the third approach. CAPT_time=RTT/2. - -[DONE] [EM] Consider whether we should delay RCVD packet printing - slightly so that CAPT packets received just slightly afterward could - be printed before the RCVD. This might make the most sense if we do - the previous feature where we show the time that a packet was - actually captured by echo server. If we did it in normal cases, it - might make it easier to compare SENT and CAPT packets, but would - also be a bit strange to see the timeline out-of-order. - -[DONE] Fix Windows rtt values. Right now Nsock does not seem to be giving - the callback at the proper time, or something. - -[DONE] Add --no-crypto to -h output. - -[DONE] Make sure nping does not allow generating packets with tcp src port or - tcp dst port 9929 (or --echo-port N, if that is set), because 1) the - echo server does not capture those packets and 2) to avoid messing up the - established side-channel tcp connection. - -[DONE] Add support for custom IP binding: if user supplies -S then - the echo side-channel connection and connections in TCP-Connect mode should be - established from that IP. This includes the echo server binding to that IP. - -[DONE] Make nping issue a warning when user supplies a payload in TCP-Connect - mode. - -[DONE] [EM] Echo server should print which interface is using to capture packets. - -[DONE] In some cases, when using nping through a VPN connection, nsi_pcap_linktype() - returns something different to DLT_EN10MB, and Nping fatals. Investigate - why this happens to nping and is not a problem for Nmap. Also, determine - why this doesn't happen all the time. What does it change between these - two?: sudo nping --udp 1.1.1.1 -g 999 -p998 - sudo nping --udp 1.1.1.1 -g 999 -p999 - The first one works, and the other one fatals with the "Currently only - Ethernet is supported." (error message @ nping.cc:1717). - - Note this also happens when Fyodor uses Nping tethering through - his cell phone (ppp0) - -[DONE] [EM] Make the server stop capturing packets when all connected clients - finish their session. - -[DONE] [EM] Some things to keep in mind for the implementation and to update - our design docs accordingly: - o Implement different "modes" for the server: complete access, - one-time-access, and restricted. - -[DONE] Do more testing on MS Windows. - -[DONE] [EM] Investigate why the echo server does not send NEP_ECHO messages when the - client sends probes at a very high rate, like in : - ./nping -c 1000 --rate 1000 --echo-client "pass" --icmp -v echo.nmap.org - -[DONE] [EM] Add echo mode to the man page - - -[DONE] [EM] Do some extensive testing of the Echo mode once it is working - to try and flesh out any bugs before merging. - -[DONE] Make Nping call nsi_delete() on pcap IODs, IODs in TCP-Connect mode and maybe - in IODs of other modes. See http://seclists.org/nmap-dev/2010/q3/587 - -[DONE] Fix bug that causes Nping to fail when sending UDP packets to a broadcast - address. More info: - -[DONE] When doing ICMP echo traceroute (with --traceroute), unless the user - supplies a custom round count (-c/--count), Nping only sends 5 packets - (default round count). This is usually not enough to reach hosts - on the internet. What should be the default behaviour? Stick with the - default round count of 5 or increment it when --traceroute is set? - - We should probably set -c 32 when --traceroute is specified, - unless user specifies their own -c explicitly. - -[DONE] Try to reduce the size of the internal buffer in the EchoHeader class. - Currenltly it allocates a big buffer that is able to hold the theoretical - maximum size of a NEP message (normal use does not require so much space). - When this is done, check if we still need to increase the stack size - in the project properties in Visual Studio. - -[DONE] [Fixed by Vasiliy Kulikov] When running Nping in ARP mode, hexdump of - ARP replies is not shown with -vvv, only for requests. Here's the output: - -sudo nping --arp 192.168.240.139 -vvv -d1 - -Starting Nping 0.5.59BETA1 ( https://nmap.org/nping ) at 2011-07-11 12:32 CEST -BPF-filter: arp and arp[6]==0x00 and arp[7]==0x02 -SENT (0.0562s) ARP who has 192.168.240.139? Tell 192.168.240.1 -0000 ff ff ff ff ff ff 00 50 56 c0 00 01 08 06 00 01 .......PV....... -0010 08 00 06 04 00 01 00 50 56 c0 00 01 c0 a8 f0 01 .......PV....... -0020 00 00 00 00 00 00 c0 a8 f0 8b .......... -RCVD (0.0568s) ARP reply 192.168.240.139 is at 00:0C:29:E4:90:CD -SENT (1.0580s) ARP who has 192.168.240.139? Tell 192.168.240.1 -0000 ff ff ff ff ff ff 00 50 56 c0 00 01 08 06 00 01 .......PV....... -0010 08 00 06 04 00 01 00 50 56 c0 00 01 c0 a8 f0 01 .......PV....... -0020 00 00 00 00 00 00 c0 a8 f0 8b .......... - - diff --git a/todo/patrick.txt b/todo/patrick.txt deleted file mode 100644 index 7508afd78..000000000 --- a/todo/patrick.txt +++ /dev/null @@ -1,77 +0,0 @@ -=== - -Currently working on: - --- LPEG in NSE. - --- HTTP Library in LPeg. - -=== - -Maybe: - --- NSE Debugger. Look at Diman's implementation: - http://seclists.org/nmap-dev/2008/q1/0228.html - http://www.keplerproject.org/remdebug/ - --- Review NSE Nsock Socket Allocation: - o Dynamically increase socket slots if nothing has been done - in the last ~5 seconds. Also decrease once traffic is working again. - This resolves any sort of socket deadlock. - --- Deadlock identification and correction: - o Add detection for deadlocks and print which threads are involved. - o use above results to make a strategy for automatic deadlock resolution. - --- Look into moving Packet Module to C. - -=== - -Done: - --- Review and Improve NSE Nsock Library. - o Move away from C pointer references and allocation over to Lua. - If a function ends in error, all the userdata will be collected. - We would otherwise need to use pcalls everywhere to clean up - and free malloc()'d memory. - o Use thread calling nsock_loop (or currently running thread) - for restoring waiting threads to the running queue. - Making a function call on a yielded thread is a hack and - could cause problems in the future. - o Get rid of the static nsock_pool and use a dynamically allocated - structure on a per-host-group basis. - o Prepare for Lua 5.2 --> Change to real errors. - --- Update NSE Book Implementation Section. - --- Added boolean operator patch. - --- Update NSE --script section (book) to include Boolean operators. - --- Fix ceil for runlevels. - --- Solve Brandon's Segfault for thread's sockets and close them when - the thread ends. - --- Change the error on finding the name of a nonexistent file in script.db - into a non-fatal warning. - --- Correct nsock_connect to unlock the socket slot if the connection fails. - --- Remove packet.hextobin and packet.bintohex. Fix scripts that used them - to instead use bin.(un)pack. - --- Commit --script-args patch and update the relevant section in the book. - --- Deadlock identification and correction: - o Release mutexes upon script death. - --- Review NSE Nsock Socket Allocation: - o Release socket locks on connection failure or timeout. - o Track active sockets in the nsock library and don't rely on - garbage collection for reallocation. - --- HTTP Caching: - o Add ability to use a proxy to http.lua. - o Test http.lua performance using local caching proxy. - o Implement a cache in http.lua. diff --git a/todo/paulino.calderon.txt b/todo/paulino.calderon.txt deleted file mode 100644 index 6f885237c..000000000 --- a/todo/paulino.calderon.txt +++ /dev/null @@ -1,4 +0,0 @@ -TODO: - --Update wiki page. --Fix: http-enum does not work on windows. UNIX paths are hardcoded into the script. It also fails when running from a directory with spaces in the name. \ No newline at end of file diff --git a/todo/sctp.txt b/todo/sctp.txt deleted file mode 100644 index 55bf04265..000000000 --- a/todo/sctp.txt +++ /dev/null @@ -1,49 +0,0 @@ -TODO.sctp $Id$ -*-text-*- - -o Further investigate SCTP functionality, as some people reported - problems (see this thread: - http://seclists.org/nmap-dev/2009/q2/0669.html) - -o Add support for UDP encapsulated SCTP (9899/udp). - Basically just wrap the SCTP packets into a UDP packet. - Think about how to add support for this to libdnet first. - See this Internet Draft by Michael Tuexen for the specs: - http://tools.ietf.org/html/draft-tuexen-sctp-udp-encaps - This is actually quite a challenging task due to the - current architecture of the scan engine. How to best - differentiate a UDP packet related to a UDP scan from a - UDP wrapped SCTP packet? How to unpack the UDP wrapped - SCTP packet in order not to duplicate a lot of code? - A good solution will be non-trivial. - -o Verify ICMP response handling for SCTP. Make sure all - ICMP types are handled in an optimal way (esp. destination - unreachable: protocol unreachable). - -o Consider removing 9899/sctp from the default port list. - 9899/udp is used for UDP encapsulated SCTP. One reason - to keep 9899/sctp is likely misconfigurations. - -o Investigate whether it makes sense to store scan state in - the itag/itsn fields for INIT scans. - -o Investigate the suitability of other SCTP chunks for port - scanning and implement more scan types if they turn out to - be worthwhile. One unverified idea is to experiment with - undefined chunk types and their first two magic bits to - provoke ERROR responses. - -o Add SCTP based service probing. - -o [Ncat] Consider implementing SCTP broker mode. - -o [NSE] Add SCTP support to NSE. - -o Investigate on differences between SCTP stacks and - implement SCTP based OS detection probes based on the - results. For example, BSD systems send the ASCII string - KAME-BSD in INIT-ACK chunks. - -o SCTP-enable scanme.nmap.org in order to make scanme.roe.ch - obsolete. - diff --git a/todo/shinnok.txt b/todo/shinnok.txt deleted file mode 100644 index b294e4254..000000000 --- a/todo/shinnok.txt +++ /dev/null @@ -1,150 +0,0 @@ -In progress: -============ - -o We should offer partial results when a host - timeouts. I (Fyodor) have been against this in the past, but maybe - the value is sufficient to be worth the maintenance headaches. Many - users have asked for this. If we do implement this, we may want to - only print results for the COMPLETED phases (e.g. host discovery, - port scanning, version detection, traceroute, NSE, etc.) Trying to - print partial results of a port scan or NSE or the like might be a - pain. And if we print some results for a host which timeouts, we - should give a very clear warning that the results for that host are - incomplete. As an example, here is someone who hacked Nmap source - code to achieve this: http://seclists.org/pen-test/2010/Mar/108. - o Another benefit would be that it would allow us to clean - up/regularize the host output code. Right now there are I think - three places where a host's final output can be printed. If, - instead, that code just looked at what information was available and - printed that out only, we could potentially isolate it in just one - place. - o This also might let us provide a feature for skipping the rest of - an Nmap phase which is going too slowly (I think that has its own - Nmap TODO item). - -Hanging(waiting for further input, etc..): -========================================== - -o Nmap *poor's man* test suite by expanding on what I already have in - /nmap-exp/shinnok/nmap-test-script. - -o NMAP reports different service results every so often with the same port. - http://seclists.org/nmap-dev/2011/q2/815 - -o Review latest revision of Marek's ncat_proxy.patch - DONE - http://seclists.org/nmap-dev/2011/q2/573 - o Commit approval pending - -Pending: -======== - -Pending (low priority): -======================= - -o E-mail nmap-dev with GProfiles /ncrack - o Create new default username list: - http://seclists.org/nmap-dev/2010/q1/798 - o Could be a SoC Ncrack task, though should prove useful for Nmap - too - o We probably want to support several lists. Like an admin/default - list like "root", "admin", "administrator", "web", "user", "test", - and also a general list which we obtain from spidering from - emails, etc. - -Potential: -========== - -COMPLETED: -========== - -o Add a --append-output option to ncat. [DONE - r25737] - -o libpcre/pcre.h - is cleared upon make distclean thus leaving the SVN - working directory dirty - http://seclists.org/nmap-dev/2011/q2/708 - -o De-duplicate code by unifying ncat_broker.c and ncat_listen.c code paths, - either as a single file in ncat_listen.c or merge duplicate code in - ncat_listen.c and keep only broker specific code in ncat_broker.c(it it's a - lot of code, otherwise ncat_listen.c would do just fine). - -o Nmap should defer address parsing in arguments until it has read - through all the args. Otherwise you get an error if you use like -S - with an IPv6 address before you put -6 in the command line. You - get a similar problem (on David's IPv6 branch) if you do "-A -6" - (but "-6 -A works properly). - -o Delve into Lua and NSE and try to write some scripts to get the hang - of it and gain a better understanding of the NSE engine in Nmap. - o Written two NSE scripts, http-reverse-ip and http-google-email that - can be found in /nmap-exp/shinnok/nse. - -o E-mail nmap-dev with QtCreator usage steps for Nmap - --- -o Ncat hangs on ssl -> REFACTORING - some refactoring left to be done to reduce code duplication - http://seclists.org/nmap-dev/2011/q2/842 - o Commit current switch/ifdef refactoring patch. - o Research code deduplication even further. - -o Ncat chat (at least in ssl mode) no longer gives the banner greeting - when I connect. This worked in r23918, but not in r24185, which is - the one running on chat.nmap.org as of 6/20/11. Verify by running - "ncat --ssl -v chat.nmap.org" - -o Pending uncompleted SSL handshakes when in --exec* listening mode make - Ncat consume 100% cpu(core/thread). - Possible solutions: - o Listen on the union of the two sets in ncat_listen.c composed of the - current set and a secondary one, ssl_pending which should include the - pending ssl hanshake sockets. - o Timeout ssl handshakes. - o Delay adding the exec output pipes to fselect/WaitForMultipleObjects - until the ssl handshake has been completed. - http://seclists.org/nmap-dev/2011/q2/988 ---- - -o Fix ncat.xml(the input for the man page) examples section. - David came up - with the final right fix on this one. - -o Ncat should close its socket and refuse further connections after the first - one, if invoked without --keep-open. That's what traditional netcat does - too. - DONE [r24197] - http://seclists.org/nmap-dev/2011/q2/944 - o Add TEST in ncat-test.pl - DONE [r24373] - -o Closing Zenmap without stopping the scan first will leave nmap running in - the process list on Windows. [r24308] - [Actually, Zenmap was unable to kill the nmap scan processes at all on - Windows] - -o Zenmap should wait for the return exit code of the nmap scanning subprocess - upon killing it(canceled scan), otherwise the subprocesses will enter a - defunct(zombie) state.[r24235] - -o Fix build_icmp_raw and build_igmp_raw filling the packet data payload - with zeroes instead of the supplied random data, when nmap is invoked - with --data-length.[r24127] - -o Investigate and document how easy it is to drop Ncat.exe by itself - on other systems and have it work. [r24242] - http://seclists.org/nmap-dev/2011/q2/1090 - - o We should also look into the dependencies of Nmap and Zenmap. - It may be instructive to look at "Portable Firefox" - (http://portableapps.com/apps/internet/firefox_portable) which is - built using open source technology from portableapps.com, or look at - "The Network Toolkit" by Cace - (http://www.cacetech.com/products/network_toolkit.html). - -o --max-conns is broken in latest svn -> fixed in r24130, other two - bugs discovered: - o --max-conns 0 kills ncat with a glibc assertion error on calloc with - zero as nmemb(??) at: - init_fdlist(&broadcast_fdlist, o.conn_limit); - o When killing the first initiated connection on --max-conns > 1 Ncat: - Ncat: Program bug: fd (5) not on list. QUITTING. - [DONE]The previous two bugs were introduced in r24130, they are now fixed - in r24193. -