lost and found ( for me ? )

Deploy Kubernetes cluster on CoreOS by using Vagrant part 01 ( run CoreOS, make a Pod )

Here are trial and logs when running Kubernetes cluster on CoreOS.

Reference

Clone the repository.
$ cd coreos-kubernetes/multi-node/vagrant/

I will run totally 5 CoreOSs as virtual machine, one controller, three working nodes, one etcd
$ cp config.rb.sample config.rb

$ cat config.rb
$update_channel="alpha"

$controller_count=1
$controller_vm_memory=512

$worker_count=3
$worker_vm_memory=1024

$etcd_count=1
$etcd_vm_memory=512

Run VMs
$ vagrant up

You can learn how vagrant configures each node, controller, worker and etcd.


$ less Vagrantfile
ETCD_CLOUD_CONFIG_PATH = File.expand_path("etcd-cloud-config.yaml")

CONTROLLER_CLOUD_CONFIG_PATH = File.expand_path("../generic/controller-install.sh")
WORKER_CLOUD_CONFIG_PATH = File.expand_path("../generic/worker-install.sh")


Etcd :  etcd-cloud-config.yaml
Controller : ../generic/controller-install.sh
Worker :  ../generic/worker-install.sh

Five VMs are running now.

Etcd : e1
Controller : c1
Working nodes :w1, w2, w3
$ vagrant status
Current machine states:

e1                        running (virtualbox)
c1                        running (virtualbox)
w1                        running (virtualbox)
w2                        running (virtualbox)
w3                        running (virtualbox)

This environment represents multiple VMs. The VMs are all listed
above with their current state. For more information about a specific
VM, run `vagrant status NAME`.

You can access to each VM via “vagrant ssh”
$ vagrant ssh c1
Last login: Tue Mar 22 14:04:16 2016 from 10.0.2.2
CoreOS alpha (991.0.0)
Failed Units: 1
 update-engine.service
core@c1 ~ $

Download kubectl on the Vagrant host so that you can manage clusters on the Vagrant host.
$ curl -O https://storage.googleapis.com/kubernetes-release/release/v1.1.8/bin/linux/amd64/kubectl

$ chmod +x kubectl
$ sudo mv kubectl /usr/local/bin/

Configure kubectl
$ export KUBECONFIG="${KUBECONFIG}:$(pwd)/kubeconfig"

$ echo ${KUBECONFIG}
:/media/hattori/logical_vol1/Vagrant_works/CoreOS_kubernetes/coreos-kubernetes/multi-node/vagrant/kubeconfig:/media/hattori/logical_vol1/Vagrant_works/CoreOS_kubernetes/coreos-kubernetes/multi-node/vagrant/kubeconfig

$ kubectl config use-context vagrant-multi
switched to context "vagrant-multi".

$ kubectl get nodes
NAME           LABELS                                STATUS    AGE
172.17.4.201   kubernetes.io/hostname=172.17.4.201   Ready     16m
172.17.4.202   kubernetes.io/hostname=172.17.4.202   Ready     16m
172.17.4.203   kubernetes.io/hostname=172.17.4.203   Ready     16m
$

w1, w2 and w3 have IP address, 172.17.4.201,202, 203.
core@w1 ~ $ ip a sh | grep inet |grep -v inet6
   inet 127.0.0.1/8 scope host lo
   inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
   inet 172.17.4.201/24 brd 172.17.4.255 scope global eth1
   inet 10.2.71.0/16 scope global flannel.1
   inet 10.2.71.1/24 scope global docker0

$ kubectl cluster-info
Kubernetes master is running at https://172.17.4.101:443
Heapster is running at https://172.17.4.101:443/api/v1/proxy/namespaces/kube-system/services/heapster
KubeDNS is running at https://172.17.4.101:443/api/v1/proxy/namespaces/kube-system/services/kube-dns
InfluxDB is running at https://172.17.4.101:443/api/v1/proxy/namespaces/kube-system/services/monitoring-influxdb

C1(controller) has 172.17.4.101.
core@c1 ~ $ ip a s | grep inet | grep -v inet6
   inet 127.0.0.1/8 scope host lo
   inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
   inet 172.17.4.101/24 brd 172.17.4.255 scope global eth1
   inet 10.2.28.0/16 scope global flannel.1
   inet 10.2.28.1/24 scope global docker0

$ kubectl get
You must specify the type of resource to get. Valid resource types include:
  * pods (aka 'po')
  * replicationcontrollers (aka 'rc')
  * daemonsets (aka 'ds')
  * services (aka 'svc')
  * events (aka 'ev')
  * nodes (aka 'no')
  * namespaces (aka 'ns')
  * secrets
  * persistentvolumes (aka 'pv')
  * persistentvolumeclaims (aka 'pvc')
  * limitranges (aka 'limits')
  * resourcequotas (aka 'quota')
  * horizontalpodautoscalers (aka 'hpa')
  * componentstatuses (aka 'cs')
  * endpoints (aka 'ep')
error: Required resource not specified.
see 'kubectl get -h' for help.

$ kubectl get pods
NAME      READY     STATUS    RESTARTS   AGE

$ kubectl get rc
CONTROLLER   CONTAINER(S)   IMAGE(S)   SELECTOR   REPLICAS   AGE

$ kubectl get svc
NAME         CLUSTER_IP   EXTERNAL_IP   PORT(S)   SELECTOR   AGE
kubernetes   10.3.0.1     <none>        443/TCP   <none>     35m
$

$ kubectl get cs
NAME                 STATUS    MESSAGE              ERROR
controller-manager   Healthy   ok                   nil
scheduler            Healthy   ok                   nil
etcd-0               Healthy   {"health": "true"}   nil

$ kubectl get ep
NAME         ENDPOINTS          AGE
kubernetes   172.17.4.101:443   47m

[ create a pod ]

I followed the instruction as below.
http://kubernetes.io/docs/user-guide/walkthrough/

Pod is a group of one or more containers.

-  nginx container

Prepare a yaml file.
$ cat pod-nginx.yaml
apiVersion: v1
kind: Pod
metadata:
 name: nginx
spec:
 containers:
 - name: nginx
   image: nginx
   ports:
   - containerPort: 80

Create a pod
$ kubectl create -f ./pod-nginx.yaml
pod "nginx" created

$ kubectl get pods
NAME      READY     STATUS    RESTARTS   AGE
nginx     0/1       Pending   0          58s

Get env of pod “nginx”
$ kubectl exec nginx env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=nginx
KUBERNETES_PORT_443_TCP_ADDR=10.3.0.1
KUBERNETES_SERVICE_HOST=10.3.0.1
KUBERNETES_SERVICE_PORT=443
KUBERNETES_PORT=tcp://10.3.0.1:443
KUBERNETES_PORT_443_TCP=tcp://10.3.0.1:443
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_PORT_443_TCP_PORT=443
NGINX_VERSION=1.9.12-1~jessie
HOME=/root

$ kubectl exec nginx ip a s
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
   link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
   inet 127.0.0.1/8 scope host lo
      valid_lft forever preferred_lft forever
   inet6 ::1/128 scope host
      valid_lft forever preferred_lft forever
8: eth0@if9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
   link/ether 02:42:0a:02:47:03 brd ff:ff:ff:ff:ff:ff
   inet 10.2.71.3/24 scope global eth0
      valid_lft forever preferred_lft forever
   inet6 fe80::42:aff:fe02:4703/64 scope link tentative dadfailed
      valid_lft forever preferred_lft forever

$ kubectl exec -ti nginx /bin/bash
root@nginx:/# exit
exit
$ kubectl exec -ti nginx /bin/bash
root@nginx:/# ip a s | grep inet  
   inet 127.0.0.1/8 scope host lo
   inet6 ::1/128 scope host
   inet 10.2.71.3/24 scope global eth0
   inet6 fe80::42:aff:fe02:4703/64 scope link tentative dadfailed
root@nginx:/# exit
exit
$

$ kubectl get pod nginx -o=yaml
apiVersion: v1
kind: Pod
metadata:
 creationTimestamp: 2016-03-22T14:35:08Z
 name: nginx
 namespace: default
 resourceVersion: "1563"
 selfLink: /api/v1/namespaces/default/pods/nginx
 uid: 46e2cb05-f03b-11e5-8365-080027226901
spec:
 containers:
 - image: nginx
   imagePullPolicy: IfNotPresent
   name: nginx
   ports:
   - containerPort: 80
     protocol: TCP
   resources: {}
   terminationMessagePath: /dev/termination-log
   volumeMounts:
   - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
     name: default-token-oexsv
     readOnly: true
 dnsPolicy: ClusterFirst
 nodeName: 172.17.4.201
 restartPolicy: Always
 serviceAccount: default
 serviceAccountName: default
 terminationGracePeriodSeconds: 30
 volumes:
 - name: default-token-oexsv
   secret:
     secretName: default-token-oexsv
status:
 conditions:
 - lastProbeTime: null
   lastTransitionTime: null
   status: "True"
   type: Ready
 containerStatuses:
 - containerID: docker://4cea5da270079e10c54eaedb9a51b85ad7745a0ee5e06705742aed036dc68133
   image: nginx
   imageID: docker://sha256:af4b3d7d5401624ed3a747dc20f88e2b5e92e0ee9954aab8f1b5724d7edeca5e
   lastState: {}
   name: nginx
   ready: true
   restartCount: 0
   state:
     running:
       startedAt: 2016-03-22T14:36:15Z
 hostIP: 172.17.4.201
 phase: Running
 podIP: 10.2.71.3
 startTime: 2016-03-22T14:35:08Z

$ kubectl delete pod nginx
pod "nginx" deleted

$ kubectl get pod
NAME      READY     STATUS    RESTARTS   AGE
$

- persistent volume

$ cat pod-redis.yaml
apiVersion: v1
kind: Pod
metadata:
 name: redis
spec:
 containers:
 - name: redis
   image: redis
   volumeMounts:
   - name: redis-persistent-storage
     mountPath: /data/redis
 volumes:
 - name: redis-persistent-storage
   emptyDir: {}


$ kubectl create -f pod-redis.yaml
pod "redis" created


$ kubectl get pod redis
NAME      READY     STATUS    RESTARTS   AGE
redis     1/1       Running   0          1m

$ kubectl exec redis env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=redis
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_443_TCP_ADDR=10.3.0.1
KUBERNETES_SERVICE_HOST=10.3.0.1
KUBERNETES_SERVICE_PORT=443
KUBERNETES_PORT=tcp://10.3.0.1:443
KUBERNETES_PORT_443_TCP=tcp://10.3.0.1:443
GOSU_VERSION=1.7
REDIS_VERSION=3.0.7
REDIS_DOWNLOAD_URL=http://download.redis.io/releases/redis-3.0.7.tar.gz
REDIS_DOWNLOAD_SHA1=e56b4b7e033ae8dbf311f9191cf6fdf3ae974d1c
HOME=/root

Log into the container and create a file.
$ kubectl exec redis -ti /bin/bash
root@redis:/data# mount | grep redis
/dev/sda9 on /data/redis type ext4 (rw,relatime,seclabel,data=ordered)
root@redis:/data#

root@redis:/data/redis# pwd
/data/redis
root@redis:/data/redis# echo hi > hi.txt
root@redis:/data/redis#

Redis container is running on w1(172.17.4.201)
$ kubectl get po redis -o=yaml


 hostIP: 172.17.4.201
 phase: Running
 podIP: 10.2.18.3
 startTime: 2016-03-23T15:12:44Z


$ vagrant ssh w1 -c "ip a s eth1"
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
   link/ether 08:00:27:1b:6f:06 brd ff:ff:ff:ff:ff:ff
   inet 172.17.4.201/24 brd 172.17.4.255 scope global eth1
      valid_lft forever preferred_lft forever
   inet6 fe80::a00:27ff:fe1b:6f06/64 scope link
      valid_lft forever preferred_lft forever

t$ vagrant ssh w1 -c "docker ps | grep redis"
df945b71accc        redis                                                "/entrypoint.sh redis"   14 minutes ago      Up 14 minutes                           k8s_redis.f4ca2bdb_redis_default_b1f464fe-f109-11e5-a453-080027226901_58f0708f
1a8e1893e84c        gcr.io/google_containers/pause:0.8.0                 "/pause"                 14 minutes ago      Up 14 minutes                           k8s_POD.6d00e006_redis_default_b1f464fe-f109-11e5-a453-080027226901_c4cc95f3
Connection to 127.0.0.1 closed.

$ kubectl delete po redis
pod "redis" deleted

- Multiple containers

This does not work in my environment, but we can learn how we define multiple containers.

$ cat pod-multi-containers.yaml
apiVersion: v1
kind: Pod
metadata:
 name: www
spec:
 containers:
 - name: nginx
   image: nginx
   volumeMounts:
   - mountPath: /srv/www
     name: www-data
     readOnly: true
 - name: git-monitor
   image: kubernetes/git-monitor
   env:
   - name: GIT_REPO
     value: http://github.com/some/repo.git
   volumeMounts:
   - mountPath: /data
     name: www-data
 volumes:
 - name: www-data
   emptyDir: {}

$ kubectl create -f pod-multi-containers.yaml
pod "www" created

$ kubectl get po
NAME      READY     STATUS           RESTARTS   AGE
www       1/2       PullImageError   0          2m

$ kubectl delete pod www
pod "www" deleted