Code documentation generators (free tools)

Source Code documentation is an often neglected  aspect of Software Engineering. In most software projects, an overview of the code structure is useful both for new developers looking at the code for the first time, as well as a reference for all developers interacting with this code.

Fortunately, there are many useful tools to help us with this issue, namely, Code Documentation Generators. These tools pick up the code’s structure through syntactic  parsing and typically augment it with information supplied by the developers directly in the source code. In the last step, a nice HTML-based reference manual is typically exported.

The following sections  present some interesting examples of this type of tool.

C/C++

Doxygen

Screenshots of doxygen documentation:

HTML:

Screen Shot 11-12-17 at 05.57 PM

Latex:

Java

Javascript

C#/VB

  • Sandcastle Help File Builder (SHFB)
    • https://github.com/EWSoftware/SHFB
    • A standalone GUI, Visual Studio integration package, and MSBuild tasks providing full configuration and extensibility for building help files with the Sandcastle tools

    • Screen Shot 06-07-17 at 10.54 AM

 

Advertisements

BitTorrent Tracker Protocol examples

HTTP Tracker

HTTP protocol is used and a typical request contains

  • info_hash
    • 20 byte sha1 hash of the bencoded form of the info value from the metainfo file
  • key
  • peer_id
    • string of length 20 which this downloader uses as its id
  • port
  • downloaded
    • total amount downloaded so far
  • left
    • number of bytes this peer still has to download
  • uploaded
    • total amount uploaded so far
  • compact
  • event
    • optional key which maps to started, completed, or stopped

Examples:

GET /announce?peer_id=aaaaaaaaaaaaaaaaaaaa&
info_hash=aaaaaaaaaaaaaaaaaaaa&
port=6881&
left=0&
downloaded=100&
uploaded=0&
compact=1
GET /announce?info_hash=%fc~6%f2%d01d%8e%f3%cd%dd%a0%1f%f7%3a%9d%ffH%cd%e3&
peer_id=-UT3480-P%a6%93%02%b4%40%27%9b%60%e9A%ed&
port=20111&uploaded=0&
downloaded=0&
left=0&
corrupt=0&
key=10E0CE47&
event=started&
numwant=200&
compact=1&
no_peer_id=1&
ip=192.168.43.188

HTTP/1.1
Host: 192.168.189.128:9000
User-Agent: uTorrent/348(110208592)(42576)
Accept-Encoding: gzip
Connection: Close

Screen Shot 10-30-17 at 03.33 PM 001Screen Shot 10-30-17 at 03.33 PM

HTTP Tracker Responses

    • Tracker responses are bencoded dictionaries.
        • if a tracker response has a key failure reason that maps to a human readable string which explains why the query failed

       

    • the response contains two typical keys:
      • interval
          • number of seconds the downloader should wait between regular rerequests

         

      • peers. peers
        • a list of peers, each peer containing
            • peer id
            • ip
            • port

           

Examples:

HTTP/1.1 200 OK

d8:intervali1800e5:peersld2:ip13:192.168.189.14:porti20111eeee

Screen Shot 10-30-17 at 03.41 PM

UDP Tracker

URLs for this protocol use the form udp://tracker:port. This type of tracker was created to improve on the overhead caused by the HTTP protocol usage. The URLs can be obtained in the metadata file for the torrent.

Possible requests supported by a UDP Tracker:

  • 0: connect
  • 1: announce
  • 2: scrape
  • 3: error

Connect Request

Before announcing, the client must obtain a connection ID (to avoid IP spoofing problems).

  • Choose a (random) transaction ID, Fill the connect input structure, Send the packet.
  • connect input
    • Offset 00 | 64-bit integer | connection_id 0x41727101980
      Offset 08 | 32-bit integer | action 0 (connect)
      Offset 12 | 32-bit integer | transaction_id

ExampleScreen Shot 10-30-17 at 04.00 PM

Connect Response

  • connect response
    • Offset 00 | 32-bit integer | action 0 (connect)
      Offset 04 | 32-bit integer | transaction_id
      Offset 08 | 64-bit integer | connection_id (random)

Announce, scrape, error

These messages are similar to the connect message, using the same semantics as the HTTP Tracker requests.

References

 

 

BitTorrent Protocol (a.k.a. Peer Protocol) examples

Here I present some examples of BitTorrent protocol interactions.

Wireshark can be used to analyze BitTorrent protocol interactions in TCP/IP.

Remember that BitTorrent’s peer protocol operates over TCP or uTP. At the time of writing, Wireshark could identify correctly a uTP connection, but unfortunately would not decode its contents as a BitTorrent protocol session. It decodes it fine for TCP/IP connections.

Message flow/sequence

screen-shot-10-21-16-at-04-34-pm

Handshake example

The Handshake message flows in both directions, this means that each peer sends an handshake message to the other.

screen-shot-10-25-16-at-03-54-pm

“Extended” message examples

In these messages we can see which extensions are supported by a peer / downloader.

uTorrent

d
1:m
d1
1:upload_only
i3e
11:lt_donthave
i7e
12:ut_holepunch
i4e
11:ut_metadata
i2e
6:ut_pex
i1e
10:ut_comment
i6e
e
13:metadata_size
i401e
1:p
i20111e
4:reqq
i255e
1:v
15:Torrent 3.4.8
2:yp
i6881e
6:yourip4:
e

qbittorent

d
1:m
d1
1:lt_donthave
i5e
11:upload_only
i2e1
1:ut_metadata
i15e
6:ut_pex
i1e
e
13:metadata_sizei401e
4:reqq
i250e
1:v
18:qBittorrent v2.9.8
6:yourip
4:
e

 

Port, Interested, Unchoke example

Screen Shot 10-12-17 at 11.34 AM

Request+Piece example

A request for a piece of a file:

Screen Shot 10-12-17 at 11.36 AM

The reply with the piece’s data contents:

Screen Shot 10-12-17 at 11.37 AM

Not Interested example

Screen Shot 10-12-17 at 11.37 AM 001

 

Downloader Peers screenshots

Usually, when a peer is connected to another one, the remote peer appears in the “Peers” tab for a torrent.

Example

screen-shot-10-19-16-at-12-18-pm

Downloader Ports configuration

uTorrent

screen-shot-10-19-16-at-12-25-pmscreen-shot-10-19-16-at-12-23-pm

 

screen-shot-10-20-16-at-07-26-pm

References

 

 

 

My new e-book “Deep VMware™ Guest Tools and Guest-Hypervisor communication” at Amazon

Just published my new e-book “Deep VMware™ Guest Tools and Guest-Hypervisor communication” at Amazon.

Check it out.

Most virtualization platforms provide some sort of mechanism of communication between the the hypervisor and its guest virtual machines. “Open VM Tools” is a set of tools that implements such communication mechanisms for VMware™ virtual machines and hypervisors. In this book we analyze each of these these tools and APIs, from high-level usage to low-level communication details, between the guest and the host. This information can be used for a better understating of what actually happens when using a guest machine with these tools. It can also be used as inspiration for using and extending guest-hypervisor communication and penetration testing.

Screen Shot 10-05-17 at 12.44 PM

cover.jpg

DDoS prevention and mitigation

There are many DDoS prevention and mitigation products. Many of these products work at the network level, filtering out malicious packets.

For example, Guard-Host states that it provides:

  • a mitigation solution based on VAC technology

    • Analyse all packets at high speed in real time
    • Vacuum your server’s incoming traffic
    • Mitigate i.e. singling out all the illegitimate IP packets, while allowing legitimate ones to pass through
  •  “To detect the attack, we use the netflow sent by the routers and analysed by the Arbor Peakflow boxes. Each router sends a summary of 1/2000 of the traffic that is actually passing through it. The Arbor Peakflow boxes analyse this and compare it to the attack signatures. If the comparison is positive, mitigation is activated within seconds.
  • The signatures analysed are based on traffic thresholds of
    • “packets per second” (pps, Kpps, Mpps, Gpps) or
    • “bits per second” (bps, Kbps, Mbps, Gbps) on certain packet types”

DDoS attack types

For example, Guard-Host acknowledges the following DDoS attack types:

DDoS Attack Types

DDoS Attack Types

Mitigation phase

In the following diagram, the packets in the red area are flagged as belonging to a DDoS attack and are thus discarded and not sent to the server under attack.

Attack detection

References

 

User Experience (UX) References

Here are some interesting references on User Experience / User Interfaces, which I personally recomend:

HAPS-High-altitude platform station

This a kind of low altitude satellite:

A high-altitude platform (HAP) is a quasi-stationary aircraft that provides means of delivering a service to a large area while staying thousands of feet above in the air for long periods of time.”

High-altitude platform station (short: HAPS) is – according to Article 1.66A of the International Telecommunication Union´s (ITU) ITU Radio Regulations (RR)[2] – defined as “a station on an object at an altitude of 20 to 50 km and at a specified, nominal, fixed point relative to the Earth”.

Traveling Ruby (multi-platform portable Ruby binaries)

Traveling Ruby consists of a set of multi-platform portable Ruby binaries, which can be used to distribute Ruby-based products and run them even in machines where Ruby is not installed. It’s very useful, as you can also use it to pack multi-platform applications.

You can check the project’s home page here:

Traveling Ruby is a project which supplies self-contained, “portable” Ruby binaries: Ruby binaries that can run on any Linux distribution and any OS X machine. It also has Windows support (with some caveats). This allows Ruby app developers to bundle these binaries with their Ruby app, so that they can distribute a single package to end users, without needing end users to first install Ruby or gems.

It can run on

  • Linux x86.
  • Linux x86_64.
  • OS X
  • Windows

 

 

Video:

https://vimeo.com/phusionnl/review/113827942/ceca7e70da

 

Code Smells detectors

A Code smell is an interesting Software Engineering concept. According to Wikipedia, a Code Smell

“refers to any symptom in the source code of a program that possibly indicates a deeper problem.[1] According to Martin Fowler, “a code smell is a surface indication that usually corresponds to a deeper problem in the system“.[2] Another way to look at smells is with respect to principles and quality:[3] “smells are certain structures in the code that indicate violation of fundamental design principles and negatively impact design quality“.

Common code smells include:

  • Class-level smells
    • Large class, Feature envy, Inappropriate intimacy, Refused bequest, Lazy class/freeloader
  • Method-level smells
    • Too many parameters, Long method, Excessively long identifiers, Excessively short identifiers, Excessive return of data
    • Excessive use of literals, Cyclomatic complexity, Downcasting
    • Orphan variable or constant class
    • Data clump

Code Smells detection tools

Some free code smell detection tools (which perform static code analysis) include:

Conclusion

Most of these smells lower your code’s quality and maintainability. Be sure to include some of these detection tools on your development processes, as well as appropriate coding standards. Automatic noncompliance detection can be accomplished by adding these tools to your build process.

You can research more tools at https://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis

References

New e-Book: “VMware™ hypervisor fingerprinting”

Just published a new e-book at Amazon.com: “VMware™ hypervisor fingerprinting”.

You can find it here:

«In this book, we show how to determine hypervisor properties by running commands in the guest operating system, without any special privileges in the host machine running the hypervisor. This can be useful for penetration testing, information gathering, determining the best software configuration for virtualization-sensitive and virtualization-aware software. Finally, we present a reporting tool that unifies all the presented methods, by running them all in sequence and gathering the information in a useful report that can be run from any guest system.»