The Compiled HTML (CHM) format was developed by Microsoft as a solution to the problem of distributing an online viewable version of documentation as a convenient single file.
At the core, CHM is just HTML, but some additional files are produced during the build process to generate a suitable table of contents and index file that can be indexed and updated so make browsing and searching the CHM content easier.
The core of the CHM process is therefore a suite a XSL stylesheets that generated the HTML and the associated TOC and index files required by the CHM standard. This process can be run on any machine.
To actually produce a CHM file, you need to use a Windows only application called HTMLHelp that compiles (and compresses) the individual HTML files and the table of contents/index files into the CHM file.
It is difficult to build the files directly on Windows. At MySQL we use a combination of assembly of the core files on a Unix system and a conversion of that source XML file and the source images on a Windows system to produce the CHM. The Windows system uses xsltproc to create the HTML files and the associated table of contents and index files, and htmlhelp to create the final CHM.