More and more companies and therefore developers are working on micro-service applications. This is great because you can develop applications that are more maintainable over time and for which you have control over the scope of your business. It may also allow you to approach container technologies and container orchestration technologies such as Kubernetes. So you have environments that allow you to deploy all these services on Kubernetes.

However, when it comes to getting on the developer's side, everyone often has their own way of doing things: their tips for launching the micro-service they are working on on their machine and testing it, perhaps by porting it to an environment to avoid having to launch it all locally. This is a case I met recently during a mission and it seemed interesting to me to answer these questions:

• Unite the environment of developers working on micro-services to have better interactions between them (and facilitate the arrival of newcomers),
• Unite the micro-service configurations to prevent developers from changing them, each according to their needs,
• Allowing you to choose "à la carte" the applications we want to run locally,
• Also possibly allow not to run some applications locally but to contact other applications on an environment directly (via port-forward),
• Finally, avoid the developer to launch 10 terminals to launch 10 micro-services and as many command lines.

In this way, everyone would have the flexibility to work in the way they want and in a simple way. For this, I created http://github.com/eko/monday, a command line tool, developed with the Go language, that allows simple use, configuration sharing between developers and flexibility on project configuration.

For a small preview of the tool, which is executed simply by typing "monday" in its terminal and choosing a project, go here:

How Monday works ?

The configuration is declared in one (or more) YAML file(s) and it is therefore possible to share it with each developer in the team. I would not detail the configuration in this article, for more details, I invite you to go to the [https://github.com/eko/monday/wiki/Configuration](Wiki page dedicated to the GitHub repository of the project).

Here is a diagram explaining Monday's principles and we will detail its operation throughout this article:

The tool consists of several components, shown here in purple:

• Runner: This is the component that allows third-party applications or services to be run locally (on your machine),
• Watcher: This component allows you to reload an application when a change is made in the code database: useful when working on micro-services in GB for example,
• Forwarder: When you do not want to launch an application locally but prefer to use the application on an environment directly, this component allows you to forward (local or remote) to Kubernetes, in SSH or simply in TCP proxy mode,
• Proxy: This component allows, both forwards and local applications to map a hostname with a private IP to allow you to unify your application configurations.

We will now go into a little more detail about each of these components in the rest of this article to understand their technical functioning.

Runner: How processes are started locally ?

The idea of this Runner component is to run several applications on your machine, to display you the logs of these on a single output and of course not to forget to stop them when you decide to terminate the Monday application.

The processes are executed via the Go package https://golang.org/pkg/os/exec/:

cmd := exec.Command(application.Executable, application.Args...)
// ...
err := cmd.Run()

When the Watcher component detects a change, it is necessary to stop this process and start a new one. Depending on the processes started, they may not have been killed by the Runner component because it would only terminate the child process. Indeed, if you take a simple example of a.sh script that would launch a command, here is what we get in terms of processes:

  PID  PPID  PGID STAT COMMAND
56503 55708 56503 S+   /bin/bash ./script.sh
56504 56503 56503 S+   sleep 20

What is important to note here is the notion of "PGID" (Process Group ID), which brings together our parent and child processes in the same group. In Gb, for the PGID to be assigned, we must add the following line to our command:

cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}

Thus, when we want to cut our process (bringing together parent and children), we will simply have to kill the process group id, in this way:

pgid, err := syscall.Getpgid(cmd.Process.Pid)
if err == nil {
    syscall.Kill(-pgid, 15)
}

All our processes are now well completed and can be restarted with peace of mind!

Proxy: How does it work?

The proxy is Monday's central brick, it allows the assignment of a hostname to a new private IP and the mapping of ports to this IP. Go unfortunately does not offer anything to manipulate network interfaces (a package https://golang.org/src/net/interface.go only displays information), so we use again the commands and ifconfig to assign an IP to the network interface.

We use the loopback interface lo0 and assign an IP for each application:

command := "ifconfig"
args := []string{"lo0", "alias", ip.String(), "up"}

err := exec.Command(command, args...).Run()
// ...

To test if a port is already used for a given IP and port, nothing could be easier, just try to open a connection on this IP and the associated port:

conn, err := net.Dial("tcp", fmt.Sprintf("%s:%s", ip.String(), port))
if err == nil {
    // No error returned, IP/port are available
    return net.IPv4(a, b, c, byte(i)), nil
}
conn.Close()
// Here, IP/Port is not available, search for another

Once the IPs and hostnames have been assigned, in case this hostname (for example: grpc-api.svc.local) is intended to point to a third-party service not launched locally, we launch a TCP proxy to read and write the remote client's responses locally, in this way:

listener, err := net.Listen("tcp", fmt.Sprintf("%s:%s", localIP, localPort))
if err != nil {
    fmt.Printf("❌  Could not create proxy listener for '%s:%s' (%s): %v\n", localIP, localPort, hostname, err)
}

for {
    client, _ := listener.Accept()
    target, _ := net.Dial("tcp", fmt.Sprintf("%s:%s", remoteHostname, proxyPort))

    // Write response to local client
    go func() {
        defer client.Close()
        defer target.Close()
        io.Copy(client, target)
    }()

    // Send request to remote client
    go func() {
        defer client.Close()
        defer target.Close()
        io.Copy(target, client)
    }()
}

Thus, we have a system that allows us to have a real two-way TCP proxy.

Forwarder: How the "local" proxy works ?

What I call the "local" proxy is actually a classic portforward, equivalent in ssh to ssh -L, allowing you to map a remote connection to a local port. On the Kubernetes side, via the tool kubectl, it is the command kubectl port-forward, allowing you to map a port of a remote service locally so that you can access it from your machine.

To do this, Monday simply uses the ssh command for SSH forwarding and for Kubernetes, we use the Go Kubernetes client:

request := restClient.Post().Resource("pods").Namespace(namespace).Name(pod.Name).SubResource("portforward")
url := url.URL{
    Scheme:   request.URL().Scheme,
    Host:     request.URL().Host,
    Path:     buildPath(request),
    RawQuery: "timeout=30s",
}

transport, upgrader, _ := spdy.RoundTripperFor(f.clientConfig)
dialer := spdy.NewDialer(upgrader, &http.Client{Transport: transport}, "POST", &url)

l := NewLogstreamer(pod.Name)
stopChannel := make(chan struct{}, 1)
readyChannel := make(chan struct{})

fw, _ := portforward.New(dialer, ports, stopChannel, readyChannel, l, l)

err = fw.ForwardPorts()

From a resource, it allows you to easily launch a port-forward locally and also offers you the possibility to use two channels readyChannel allowing you to identify when the forward is functional and stopChannel when the connection is broken: you can thus easily relaunch the connection.

Forwarder: How does the "remote" proxy work?

Now, a little more complex: I will describe how we do remote forwarding: it is about transmitting incoming traffic from a Kubernetes service to your machine locally. There are two notions that we will use and that are to be known for this purpose:

• The SSH remote-forward (ssh -R), unlike the local-forward (ssh -L), it allows to direct traffic from the remote to the client connection, for a given port,
• To be able to do this SSH remote-forward, we will deploy the proxy image https://github.com/eko/monday/blob/master/docker-proxy/Dockerfile on your Kubernetes service.

Here is the principle of remote-forward to Kubernetes:

Idea is basically to update the Kubernetes deployment to set the ekofr/monday-proxy Docker image that will open the SSH port and allow you to forward it locally. Once it is available locally, Monday just have to run a SSH remote-forward.

Finally, a really cool thing with the Kubernetes Go client is that it exposes a lot of interfaces so it's easy to mock the Kubernetes Go client in unit tests. Here is a quick example:

deploymentInterface := &clientmocks.DeploymentInterface{}
deploymentInterface.On("List", metav1.ListOptions{LabelSelector: "app=my-test-app"}).
    Return(&appsv1.DeploymentList{
        Items: []appsv1.Deployment{},
    })

appsV1Interface := &clientmocks.AppsV1Interface{}
appsV1Interface.On("Deployments", "backend").
    Return(deploymentInterface)

clientSetMock := &clientmocks.Interface{}
clientSetMock.On("AppsV1").
    Return(appsV1Interface)

Conclusion

Building this tool was really fun for me as I learn some new things from unix processus management with Go and also about the Kubernetes Go client. I would be really happy if it could be interesting for you to adopt Monday on your development team.

Do not hesitate to contact me!