Linux Mint: Compile and install the Go compiler from source

Objective

Set up bootstrapping. Build the latest version of the Go compiler with C bridge mode support from it's sources.

Motivation

By coincidence, I stumbled over this interesting tutorial about the programming language Go.
Since i was always interested to play around with that language, I took the opportunity, to try it out. As I usually want to use the latest compiler-version, I thought, it would be a good idea, that I do not use the Go-Installer, but compile the sources by myself from scratch. Unfortunately, the latest Go compiler, cannot be compiled with C support, when there is not already a Go compiler installed in the system. Therefore I also had to install an older Go compiler for booststrapping first.

Prerequisites

• Linux Mint 18.3 Sylvia - Cinnamon (64 Bit)
• GCC 7.3.0 (build essential or update-alternatives)

Solution

I decided to do the whole build and temporary stuff within the "Downloads" folder in my home account.
The boostrap toolchain will reside in the sub-directory "gobootstrap" within the "Downloads" folder.

Setup the bootstrap toolchain

Open a terminal. Download and install the latest Go bootstrap toolchain.

$> mkdir -p Downloads/goboostrap
$> cd Downloads/gobootstrap
$> wget https://dl.google.com/go/go1.4-bootstrap-20171003.tar.gz

you'll see some output similar to this:

--2018-06-23 17:58:49-- https://dl.google.com/go/go1.4-bootstrap-20171003.tar.gz
Resolving dl.google.com (dl.google.com)... 216.58.207.46, 2a00:1450:4001:824::200e
Connecting to dl.google.com (dl.google.com)|216.58.207.46|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11009739 (10M) [application/octet-stream]
Saving to: ‘go1.4-bootstrap-20171003.tar.gz.1’

go1.4-bootstrap-20171003.tar.gz.1 100%[==========================================================================>] 10,50M 5,23MB/s in 2,0s

2018-06-23 17:58:51 (5,23 MB/s) - ‘go1.4-bootstrap-20171003.tar.gz.1’ saved [11009739/11009739]

Now you have to unpack the downloaded toolchain package:

$> tar -xvzf go1.4-bootstrap-20171003.tar.gz

The whole stuff is unpacked into a new sub-directory "go"

go/.gitattributes
go/.gitignore
[...]
go/src/cmd/5g/gg.h
go/src/cmd/5g/ggen.c
go/src/cmd/5g/gobj.c
go/src/cmd/5g/gsubr.c
[...]
go/test/varerr.go
go/test/varinit.go
go/test/zerodivide.go

Change directory into the "./go/src" and build the bootstrap toolchain.
Observe: This step requires you to have already a functional GCC compiler present on your system.
If not already done: To set-up the GCC 7.3.0 on your system see Install multiple versions of GCC on your system

$> CGO_ENABLED=0 ./make.bash

# Building C bootstrap tool.
cmd/dist

# Building compilers and Go bootstrap tool for host, linux/amd64.
lib9
[...]
# Building packages and commands for linux/amd64.
runtime
errors
sync/atomic
[...]
cmd/pprof
net/rpc
net/http/fcgi
net/rpc/jsonrpc

Finally, the toolchain build is finished.

Compile the Go compiler

Before you compile the compiler, step back to the "Downloads" folder and download the latest source of the go compiler.

$> cd ~/Downloads
$> wget https://dl.google.com/go/go1.10.3.src.tar.gz

--2018-06-23 18:29:13-- https://dl.google.com/go/go1.10.3.src.tar.gz
Resolving dl.google.com (dl.google.com)... 216.58.207.78, 2a00:1450:4001:825::200e
Connecting to dl.google.com (dl.google.com)|216.58.207.78|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 18323736 (17M) [application/octet-stream]
Saving to: ‘go1.10.3.src.tar.gz’

go1.10.3.src.tar.gz 100%[==========================================================================>] 17,47M 5,94MB/s in 2,9s

2018-06-23 18:29:16 (5,94 MB/s) - ‘go1.10.3.src.tar.gz’ saved [18323736/18323736]

Now, unpack the sources, like it was already done, with the toolchain package.

$> tar -xvzf go1.10.3.src.tar.gz

go/
go/AUTHORS
go/CONTRIBUTING.md
[...]
go/src/runtime/closure_test.go
go/src/runtime/compiler.go
go/src/runtime/complex.go
[...]
go/test/varinit.go
go/test/writebarrier.go
go/test/zerodivide.go

Again, step into the "src" directory and build the compiler, using the bootstrap toolchain.
This step may take a while, depending on the performance of your computer.

$> cd go/src
$> GOROOT_BOOTSTRAP=~/Downloads/gobootstrap/go ./all.bash

Building Go cmd/dist using /home/cschmidt/Downloads/gobootstrap/go.
Building Go toolchain1 using /home/cschmidt/Downloads/gobootstrap/go.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for linux/amd64.

##### Testing packages.
ok archive/tar 0.051s
ok archive/zip 1.164s
ok bufio 0.186s
ok bytes 0.686s
ok compress/bzip2 0.132s
[...]
ok cmd/vendor/golang.org/x/arch/x86/x86asm 0.213s
ok cmd/vet 3.946s
ok cmd/vet/internal/cfg 0.033s

##### GOMAXPROCS=2 runtime -cpu=1,2,4 -quick
ok runtime 14.067s

##### cmd/go terminal test
PASS
ok _/home/cschmidt/Downloads/go/src/cmd/go/testdata/testterminal18153 0.001s

##### Testing without libgcc.
ok crypto/x509 1.016s
ok net 0.031s
ok os/user 0.038s

[...]

##### API check
Go version is "go1.10.3", ignoring -next /home/cschmidt/Downloads/go/api/next.txt

ALL TESTS PASSED
---
Installed Go for linux/amd64 in /home/cschmidt/Downloads/go
Installed commands in /home/cschmidt/Downloads/go/bin
*** You need to add /home/cschmidt/Downloads/go/bin to your PATH.

As I didn't want to have the Go compiler installed in my "Downloads" folder I simply move it directly into my home-account.

$> cd ~/Downloads
$> mv go ~/

Let's try to call the Go compiler

$> go

The program 'go' is currently not installed. You can install it by typing:
sudo apt install golang-go

Ouwww! What went wrong? -- Nothing!

I forgot to extend the PATH variable of my environment as mentioned by the hint, given after compilation.

To do so, I add the following line to my "~/.bashrc".

$> echo 'PATH=$PATH:$HOME/go/bin # Add go compiler' >> ~/.bashrc

Observe: Double-check, you use single quotes instead of double quotes here, otherwise, the bash will already expand the "$PATH" variable here and append it's content to your "~/.bashrc".

Once, again:

$> go

Go is a tool for managing Go source code.

Usage:

go command [arguments]
[...]

Yeah, finally done.
Just to clean up the mess within the "Downloads" directory by just deleting everything, I do not need anymore.

References:

1. Go - Environment Setup
2. Installing Go from source

Linux Mint: Build your own debian package of cmake

Objective

I wanted to use the newest available version of CMake (version 3.12.0-rc1) on Linux Mint 18.3 Sylvia.

Motivation

My company started using CMake as a Meta-Build-System in combination with Visual Studio 2017 in a brand new software project. Because of this fact, I had the opportunity, to attend a Modern CMake seminar at Eclipseina GmbH, covering most features of Modern CMake.
As Visual Studio 2017 comes with a CMake-component of version 3.10.0 already, I wanted at least to be able to use the same version of CMake on my Linux Mint 18.3 at home.
Modern CMake requires at least CMake version 3.x.

Unfortunately, the repository of Linux Mint 18.3 only supports a Debian package installer for CMake 3.5.1. The homepage of CMake at cmake.org only offers an install script, without uninstaller. I don't want to pollute my system with early access versions of software packages that I cannot clearly uninstall later.
In contrary, I wanted to be able to install and uninstall any version of CMake. Therefore I needed to build my own Debian install package (*.deb) for CMake.

Prerequisites

• Linux Mint 18.3 Sylvia - Cinnamon (64 Bit)
• GCC 7.3.0 (build essential or update-alternatives)

Solution

Install CMake locally

Open a terminal. Download and install the CMake installer script:

$> mkdir Downloads
$> cd Downloads
$> wget https://cmake.org/files/v3.12/cmake-3.12.0-rc1-Linux-x86_64.sh

you'll see some output like this:

--2018-06-19 22:58:04-- https://cmake.org/files/v3.12/cmake-3.12.0-rc1-Linux-x86_64.sh
Resolving cmake.org (cmake.org)... 66.194.253.19
Connecting to cmake.org (cmake.org)|66.194.253.19|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 30260259 (29M) [text/x-sh]
Saving to: ‘cmake-3.12.0-rc1-Linux-x86_64.sh.1’

cmake-3.12.0-rc1-Linux-x86_64.sh 100%[==========================================================>] 28,86M 6,02MB/s in 5,3s

2018-06-19 22:58:10 (5,41 MB/s) - ‘cmake-3.12.0-rc1-Linux-x86_64.sh.1’ saved [30260259/30260259]

Now, set the executable flag for the downloaded script and start the temporary local install as normal user

$> chmod u+x cmake-3.12.0-rc1-Linux-x86_64.sh
$> ./cmake-3.12.0-rc1-Linux-x86_64.sh

CMake Installer Version: 3.12.0-rc1, Copyright (c) Kitware
This is a self-extracting archive.
The archive will be extracted to: /home/cschmidt/Downloads

If you want to stop extracting, please press .
CMake - Cross Platform Makefile Generator
Copyright 2000-2018 Kitware, Inc. and Contributors
All rights reserved.

[...]
Do you accept the license? [yN]:

Accept the license, by typing 'y'.

By default the CMake will be installed in:
"/home/cschmidt/Downloads/cmake-3.12.0-rc1-Linux-x86_64"
Do you want to include the subdirectory cmake-3.12.0-rc1-Linux-x86_64?
Saying no will install in: "/home/cschmidt/Downloads" [Yn]:

Accept the default path, by typing 'Y'.

Using target directory: /home/cschmidt/Downloads/cmake-3.12.0-rc1-Linux-x86_64
Extracting, please wait...

Unpacking finished successfully

To be able to use the locally installed CMake, you must add it's binary directory to your environment path:
(Of course, you must use the path from above that reflects your install directory and add a "/bin" path-component here)

$> PATH=$PATH:/home/cschmidt/Downloads/cmake-3.12.0-rc1-Linux-x86_64/bin

Download and extract the CMake source package

$> wget https://cmake.org/files/v3.12/cmake-3.12.0-rc1.tar.gz

--2018-06-19 23:15:38-- https://cmake.org/files/v3.12/cmake-3.12.0-rc1.tar.gz
Resolving cmake.org (cmake.org)... 66.194.253.19
Connecting to cmake.org (cmake.org)|66.194.253.19|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 8089283 (7,7M) [application/x-gzip]
Saving to: ‘cmake-3.12.0-rc1.tar.gz.1’

cmake-3.12.0-rc1.tar.gz.1 100%[==========================================================>] 7,71M 3,46MB/s in 2,2s

2018-06-19 23:15:42 (3,46 MB/s) - ‘cmake-3.12.0-rc1.tar.gz.1’ saved [8089283/8089283]

Extract the source tar-gz package:

$> tar -xvzf cmake-3.12.0-rc1.tar.gz

cmake-3.12.0-rc1/.clang-format
cmake-3.12.0-rc1/.clang-tidy
cmake-3.12.0-rc1/Auxiliary/
cmake-3.12.0-rc1/Auxiliary/bash-completion/
cmake-3.12.0-rc1/Auxiliary/bash-completion/cmake
cmake-3.12.0-rc1/Auxiliary/bash-completion/CMakeLists.txt
[...]
cmake-3.12.0-rc1/Utilities/Sphinx/static/cmake-favicon.ico
cmake-3.12.0-rc1/Utilities/Sphinx/static/cmake-logo-16.png
cmake-3.12.0-rc1/Utilities/Sphinx/static/cmake.css
cmake-3.12.0-rc1/Utilities/Sphinx/templates/
cmake-3.12.0-rc1/Utilities/Sphinx/templates/layout.html

Compile CMake from source using your temporary CMake installation

$> cd cmake-3.12.0-rc1
$> mkdir build
$> cd build/
$> cmake ..

CMake does some checks of your system and builds the binaries from source, which takes a while ...

-- The C compiler identification is GNU 7.3.0
-- The CXX compiler identification is GNU 7.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
[...]
-- Performing Test run_inlines_hidden_test
-- Performing Test run_inlines_hidden_test - Success
-- Configuring done
-- Generating done
-- Build files have been written to: /home/cschmidt/Downloads/cmake-3.12.0-rc1/build
cschmidt@gimli:~/Downloads/cmake-3.12.0-rc1/build$ make
Scanning dependencies of target cmsys_c
[ 0%] Building C object Source/kwsys/CMakeFiles/cmsys_c.dir/ProcessUNIX.c.o
[ 0%] Building C object Source/kwsys/CMakeFiles/cmsys_c.dir/Base64.c.o
[...]
[ 1%] Building C object Source/kwsys/CMakeFiles/cmsys_c.dir/String.c.o
[ 1%] Linking C static library libcmsys_c.a
[...]
Scanning dependencies of target foo
[100%] Building CXX object Tests/FindPackageModeMakefileTest/CMakeFiles/foo.dir/foo.cpp.o
[100%] Linking CXX static library libfoo.a
[100%] Built target foo

Build the Debian package (*.deb)

If checkinstall is not installed on your machine, you can install it via:

$> sudo apt-get install checkinstall

On mine, it's already available, therefore...

Reading package lists... Done
Building dependency tree
Reading state information... Done
checkinstall is already the newest version (1.6.2-4ubuntu1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Normally checkinstall needs to be run as root and does not only build the package, but also install the software.
To just build the package, without root privileges and without automatically installing it, we have to run checkinstall using fakeroot.

$> fakeroot checkinstall --install=no --fstrans=yes

checkinstall 1.6.2, Copyright 2009 Felipe Eduardo Sanchez Diaz Duran
This software is released under the GNU GPL.


The package documentation directory ./doc-pak does not exist.
Should I create a default set of package docs? [y]: y

Confirm the question with 'y'.

Preparing package documentation...OK

*** No known documentation files were found. The new package
*** won't include a documentation directory.

*****************************************
**** Debian package creation selected ***
*****************************************

This package will be built according to these values:

0 - Maintainer: [ cschmidt@gimli ]
1 - Summary: [ CMake Release Candidate (3.12.0-rc1) ]
2 - Name: [ build ]
3 - Version: [ 20180618 ]
4 - Release: [ 1 ]
5 - License: [ GPL ]
6 - Group: [ checkinstall ]
7 - Architecture: [ amd64 ]
8 - Source location: [ build ]
9 - Alternate source location: [ ]
10 - Requires: [ ]
11 - Provides: [ build ]
12 - Conflicts: [ ]
13 - Replaces: [ ]

Now you have the opportunity to change some meta-data, e.g. name and URL:

Enter a number to change any of them or press ENTER to continue: 0
Enter the maintainer's name and e-mail address:
>> cwschmidt

Enter a number to change any of them or press ENTER to continue: 9
Enter the alternate source location:
>> https://cmake.org/files/v3.12/cmake-3.12.0-rc1.tar.gz

This package will be built according to these values:

0 - Maintainer: [ cwschmidt ]
1 - Summary: [ CMake Release Candidate (3.12.0-rc1) ]
2 - Name: [ build ]
3 - Version: [ 20180618 ]
4 - Release: [ 1 ]
5 - License: [ GPL ]
6 - Group: [ checkinstall ]
7 - Architecture: [ amd64 ]
8 - Source location: [ build ]
9 - Alternate source location: [ https://cmake.org/files/v3.12/cmake-3.12.0-rc1.tar.gz ]
10 - Requires: [ ]
11 - Provides: [ build ]
12 - Conflicts: [ ]
13 - Replaces: [ ]

Enter a number to change any of them or press ENTER to continue:

Finally, press Enter to continue

Installing with make install...

========================= Installation results ===========================
[ 1%] Built target cmsys_c
[ 2%] Built target cmsysTestsC
[ 4%] Built target cmsys
[...]
[100%] Built target pseudo_tidy
[100%] Built target pseudo_cppcheck
[100%] Built target foo
Install the project...
-- Install configuration: ""
-- Installing: /usr/local/doc/cmake-3.12/Copyright.txt
-- Installing: /usr/local/share/cmake-3.12/Help
-- Installing: /usr/local/share/cmake-3.12/Help/prop_dir
-- Installing: /usr/local/share/cmake-3.12/Help/prop_dir/VS_GLOBAL_SECTION_PRE_section.rst
[...]
-- Installing: /usr/local/share/cmake-3.12/Modules
-- Installing: /usr/local/share/cmake-3.12/Modules/FindCurses.cmake
-- Installing: /usr/local/share/cmake-3.12/Modules/FindWget.cmake
-- Installing: /usr/local/share/cmake-3.12/Modules/FindAVIFile.cmake
[...]
-- Installing: /usr/local/bin/cmake
-- Installing: /usr/local/bin/ctest
-- Installing: /usr/local/bin/cpack
[...]
-- Installing: /usr/local/share/cmake-3.12/completions/cmake
-- Installing: /usr/local/share/cmake-3.12/completions/cpack
-- Installing: /usr/local/share/cmake-3.12/completions/ctest

======================== Installation successful ==========================

Some of the files created by the installation are inside the home directory: /home

You probably don't want them to be included in the package.
Do you want me to list them? [n]: y
Should I exclude them from the package? (Saying yes is a good idea) [n]: y

You were ask, to exclude the files that were placed in your home-directory.
To inspect the list, answer the first question with 'y'.
Answer the second question to exclude the files in the home-directory with 'y'.

Copying files to the temporary directory...OK

Stripping ELF binaries and libraries...OK

Compressing man pages...OK

Building file list...OK

Building Debian package...OK

NOTE: The package will not be installed

Erasing temporary files...OK

Writing backup package...OK
OK

Deleting temp dir...OK


**********************************************************************

Done. The new package has been saved to

/home/cschmidt/Downloads/cmake-3.12.0-rc1/build/build_20180618-1_amd64.deb
You can install it in your system anytime using:

dpkg -i build_20180618-1_amd64.deb

**********************************************************************

Finished. You can install the newly created package, by typing

$> sudo dpkg -i build_20180618-1_amd64.deb

or with your debian package manager with a double-click on the file "build_20180618-1_amd64.deb".

References:

1. cmake.org
2. How do I install the latest version of cmake from the command line?
3. checkinstall source code inside home directory