Properly return negative on error from vsnprintf

Our implementation of vsnprintf for systems where it is missing did not
correctly return a negative value on error, instead returning the size
passed in. We got this code from tcpdump/libpcap, and it was wrong
there, too, though their latest master branch has removed it in favor of
requiring a C99 compiler (C99 guarantees vsnprintf).

This should remove a LGTM code analysis finding (See #1834) of
cpp/constant-comparison in Ncat because we were checking for a negative
return from Snprintf, which would never occur.

nbase/snprintf.c

@@ -639,7 +639,7 @@ vsnprintf (char *str, size_t sz, const char *format, va_list args)
   ret = xyzprintf (&state, format, args);
   *state.s = '\0';
   if (ret)
-    return sz;
+    return -1;
   else
     return state.s - state.str;
 }