The pipeline logs

Json support

For purist, there is a Json output available with pipelight logs --json.

The default log output is the following pretty tree output.

Pipeline status (global status)

A pipeline can be either running, succeeded, failed or aborted.

Other existing status are for internal purpopse and are only discussed in the source code.

[g@ku ~] 
● running-Fri, 5 Jul 2023 16:52:58 +0200
branch: dev
action: pre-commit
commit: 5f5fe3fcac3e46117fcbfr
pipeline: vitest_unit_test(6s219ms)
● succeeded-Fri, 5 Jul 2023 16:52:58 +0200
branch: dev
action: pre-commit
commit: 5f5fe3fcac3e46117fcbfr
pipeline: vitest_unit_test(6s219ms)
● failed-Fri, 5 Jul 2023 16:52:58 +0200
branch: dev
action: pre-commit
commit: 5f5fe3fcac3e46117fcbfr
pipeline: vitest_unit_test(6s219ms)
● aborted-Fri, 5 Jul 2023 16:52:58 +0200
branch: dev
action: pre-commit
commit: 5f5fe3fcac3e46117fcbfr
pipeline: vitest_unit_test(6s219ms)

Running status is when a pipeline execution isn't finished yet.

Succeeded status is when a command of the pipeline normally succeeded with an exit status.

Failed status is when a command of the pipeline normally failed with an exit status (as opposed to aborted).

Aborted status means that something unexpected interupted the pipeline execution.

It can be due to:

a resource outage.
a linux signal like SIGKILL or SIGTERM.
a Ctrl-C on a running attached pipeline.
litteraly everything that can abrubtly stop a process execution (coffee on keyboard, angry mother...)

Corrupted logs

Corrupted logs are rare but can occur on unsanitized aborted pipelines log files or on a concurrent Read and Write operation.

They are actually removed from the log output (with a discrete warning message). I am working on a way to recover them when possible and flag them as corrupted.

Verbosity levels

A pipeline has 5 verbosity levels.

Error level (default)

The first level displays global informations.

pipeline status
date it was triggered at
environment in which it was triggered, which includes:
- branch, or tag name if the project has a git repository.
- action that triggered the pipeline execution
- the commit number if the project has a git repository.
pipeline name
whole pipeline execution time

[g@ku ~] 
● running-Fri, 5 Jul 2023 16:52:58 +0200
branch: dev
action: pre-commit
commit: 5f5fe3fcac3e46117fcbfr
pipeline: vitest_unit_test(6s219ms)

Warn level (-v)

The second level shows additional informations about the inside of the pipeline. it adds:

steps names
steps execution time

[g@ku ~] 
● running-Fri, 5 Jul 2023 16:52:58 +0200
branch: dev
action: pre-commit
commit: 5f5fe3fcac3e46117fcbfr
pipeline: vitest_unit_test(6s219ms)
step: build_files(4s719ms)
step: build image itsdizygote.com/front:production(1s489ms)
step: send image itsdizygote.com/front:production to remote
step: clean containers production.front.itsdizygote.com
step: run containers production.front.itsdizygote.com

Info level (-vv)

Adds:

command list (process stdin)
command execution time

[g@ku ~] 
● running-Fri, 5 Jul 2023 16:52:58 +0200
branch: dev
action: pre-commit
commit: 5f5fe3fcac3e46117fcbfr
pipeline: vitest_unit_test(6s219ms)
step: build_files(4s719ms)pnpm install(989ms)
vite build(3s889ms)
step: build image itsdizygote.com/front:production(1s489ms)docker build  --tag itsdizygote.com/front:production  --file .docker/Dockerfile.front   .(1s489ms)
step: send image itsdizygote.com/front:production to remotedocker save itsdizygote.com/front:production | ssh -o TCPKeepAlive=no -C linode "docker load"
step: clean containers production.front.itsdizygote.comssh -o TCPKeepAlive=no -C linode "docker container stop production.front.itsdizygote.com && docker container rm production.front.itsdizygote.com"
step: run containers production.front.itsdizygote.comssh -o TCPKeepAlive=no -C linode "docker run  --detach  --publish 127.0.0.1:9280:3000  --name production.front.itsdizygote.com  itsdizygote.com/front:production"

Debug level (-vvv)

Diplay the command output

stdout on success
or
stderr on failure

[g@ku ~] 
● running-Fri, 5 Jul 2023 16:52:58 +0200
branch: dev
action: pre-commit
commit: 5f5fe3fcac3e46117fcbfr
pipeline: vitest_unit_test(6s219ms)
step: build_files(4s719ms)pnpm install(989ms)Lockfile is up to date, resolution step is skipped
Already up to date

Done in 924ms
vite build(3s889ms)vite v4.3.1 building for production...
transforming...

🌼 daisyUI 3.0.20
https://daisyui.com
╰╮
 ╰─ themes are enabled. You can add more themes or make your own theme:
      https://daisyui.com/docs/themes

[32m    ❤︎ Support daisyUI[0m: https://opencollective.com/daisyui[0m

✓ 669 modules transformed.
rendering chunks...
computing gzip size...
dist/index.html                                  0.45 kB │ gzip:  0.30 kB
dist/assets/product-bbe8489e.css                 0.33 kB │ gzip:  0.20 kB
dist/assets/products-4aa53b00.css                0.36 kB │ gzip:  0.17 kB
dist/assets/collection-c3ae63aa.css              0.36 kB │ gzip:  0.17 kB
dist/assets/index-dc1b6ad3.css                  58.72 kB │ gzip: 10.54 kB
dist/assets/dynamic-import-helper-be004503.js    0.25 kB │ gzip:  0.20 kB
dist/assets/home-cf9929fe.js                     0.33 kB │ gzip:  0.24 kB
dist/assets/products-3c507332.js                 0.59 kB │ gzip:  0.38 kB
dist/assets/about-354f315b.js                    0.71 kB │ gzip:  0.38 kB
dist/assets/legal-8c08ca70.js                    0.71 kB │ gzip:  0.39 kB
dist/assets/collection-862590ba.js               0.71 kB │ gzip:  0.45 kB
dist/assets/product-67ba29c0.js                  1.44 kB │ gzip:  0.74 kB
dist/assets/about.en-4feec2fd.js                 1.48 kB │ gzip:  0.86 kB
dist/assets/about.fr-fe09bafb.js                 1.60 kB │ gzip:  0.98 kB
dist/assets/legal.en-b572a7b4.js                11.36 kB │ gzip:  4.33 kB
dist/assets/legal.fr-64c64e93.js                12.28 kB │ gzip:  4.78 kB
dist/assets/index-0b68c6ef.js                  161.97 kB │ gzip: 60.82 kB
✓ built in 3.24s
step: build image itsdizygote.com/front:production(1s489ms)docker build  --tag itsdizygote.com/front:production  --file .docker/Dockerfile.front   .(1s489ms)
step: send image itsdizygote.com/front:production to remotedocker save itsdizygote.com/front:production | ssh -o TCPKeepAlive=no -C linode "docker load"
step: clean containers production.front.itsdizygote.comssh -o TCPKeepAlive=no -C linode "docker container stop production.front.itsdizygote.com && docker container rm production.front.itsdizygote.com"
step: run containers production.front.itsdizygote.comssh -o TCPKeepAlive=no -C linode "docker run  --detach  --publish 127.0.0.1:9280:3000  --name production.front.itsdizygote.com  itsdizygote.com/front:production"

Trace level (-vvvv)

TIP

Some tools such as linters, docker, nixos... usually print their activity/logs to stderr. You may want to use the trace level log to checkout your command progress.

Displays every command output.

stdout
stderr

Some processes default output on stderr, and doesn't display much on success. This log level is practical if you want to see what happenned or what happens:

when you run tests (cargo run tests)
when you linte files (deno lint)
on a docker command (docker build)

[g@ku ~] 
● running-Fri, 5 Jul 2023 16:52:58 +0200
branch: dev
action: pre-commit
commit: 5f5fe3fcac3e46117fcbfr
pipeline: vitest_unit_test(6s219ms)
step: build_files(4s719ms)pnpm install(989ms)stdout: Lockfile is up to date, resolution step is skipped
Already up to date

Done in 924ms
stderr: 
vite build(3s889ms)stdout: vite v4.3.1 building for production...
transforming...

🌼 daisyUI 3.0.20
https://daisyui.com
╰╮
 ╰─ themes are enabled. You can add more themes or make your own theme:
      https://daisyui.com/docs/themes

[32m    ❤︎ Support daisyUI[0m: https://opencollective.com/daisyui[0m

✓ 669 modules transformed.
rendering chunks...
computing gzip size...
dist/index.html                                  0.45 kB │ gzip:  0.30 kB
dist/assets/product-bbe8489e.css                 0.33 kB │ gzip:  0.20 kB
dist/assets/products-4aa53b00.css                0.36 kB │ gzip:  0.17 kB
dist/assets/collection-c3ae63aa.css              0.36 kB │ gzip:  0.17 kB
dist/assets/index-dc1b6ad3.css                  58.72 kB │ gzip: 10.54 kB
dist/assets/dynamic-import-helper-be004503.js    0.25 kB │ gzip:  0.20 kB
dist/assets/home-cf9929fe.js                     0.33 kB │ gzip:  0.24 kB
dist/assets/products-3c507332.js                 0.59 kB │ gzip:  0.38 kB
dist/assets/about-354f315b.js                    0.71 kB │ gzip:  0.38 kB
dist/assets/legal-8c08ca70.js                    0.71 kB │ gzip:  0.39 kB
dist/assets/collection-862590ba.js               0.71 kB │ gzip:  0.45 kB
dist/assets/product-67ba29c0.js                  1.44 kB │ gzip:  0.74 kB
dist/assets/about.en-4feec2fd.js                 1.48 kB │ gzip:  0.86 kB
dist/assets/about.fr-fe09bafb.js                 1.60 kB │ gzip:  0.98 kB
dist/assets/legal.en-b572a7b4.js                11.36 kB │ gzip:  4.33 kB
dist/assets/legal.fr-64c64e93.js                12.28 kB │ gzip:  4.78 kB
dist/assets/index-0b68c6ef.js                  161.97 kB │ gzip: 60.82 kB
✓ built in 3.24s
stderr: 
step: build image itsdizygote.com/front:production(1s489ms)docker build  --tag itsdizygote.com/front:production  --file .docker/Dockerfile.front   .(1s489ms)stdout: 
stderr: #0 building with "default" instance using docker driver

#1 [internal] load .dockerignore
#1 transferring context: 2B done
#1 DONE 0.0s

#2 [internal] load build definition from Dockerfile.front
#2 transferring dockerfile: 277B done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/archlinux:latest
#3 DONE 1.2s

#4 [1/7] FROM docker.io/library/archlinux:latest@sha256:fe6b55ecfcfe638aa13560cdc66eb2ad9bd6c4f6684d27d0d2a3816cf2882a11
#4 DONE 0.0s

#5 [internal] load build context
#5 transferring context: 5.68MB 0.0s done
#5 DONE 0.0s

#6 [2/7] RUN pacman -Sy
#6 CACHED

#7 [4/7] RUN npm add -g serve
#7 CACHED

#8 [3/7] RUN pacman -S --noconfirm npm
#8 CACHED

#9 [5/7] RUN mkdir app
#9 CACHED

#10 [6/7] WORKDIR /app
#10 CACHED

#11 [7/7] COPY ./dist .
#11 CACHED

#12 exporting to image
#12 exporting layers done
#12 writing image sha256:f0547343647bb06a2489c4d25cb44592d992dcada04a7e3566fee95190259fa4 done
#12 naming to itsdizygote.com/front:production done
#12 DONE 0.0s
step: send image itsdizygote.com/front:production to remotedocker save itsdizygote.com/front:production | ssh -o TCPKeepAlive=no -C linode "docker load"stdout: 
stderr: 
step: clean containers production.front.itsdizygote.comssh -o TCPKeepAlive=no -C linode "docker container stop production.front.itsdizygote.com && docker container rm production.front.itsdizygote.com"stdout: 
stderr: 
step: run containers production.front.itsdizygote.comssh -o TCPKeepAlive=no -C linode "docker run  --detach  --publish 127.0.0.1:9280:3000  --name production.front.itsdizygote.com  itsdizygote.com/front:production"stdout: 
stderr:

Raw logs (Json)

Logs are stored in your project root directory in the .pipelight/logs folder in JSON format. You can checkout raw logs directly by inspecting the generated files or with the following commands:

pipelight logs --json | jq
pipelight logs --json | jq -C | less

pipelight logs --json | jq
pipelight logs --json | jq -C | less

Other commands

You can inspect logs by pipeline name.

pipelight logs <pipeline_name>

pipelight logs <pipeline_name>

Keep a colorful output in pagers.

pipelight logs --color always | less

pipelight logs --color always | less

Print colorful raw json logs in pagers (debugging)

pipelight logs --json | jq -C | less

pipelight logs --json | jq -C | less

Prune/Remove logs

As of today, no log rotation has been implemented. The best way to remove corrupted logs or to clean the log directory is with the following commands.

pipelight logs rm

pipelight logs rm

Which does not erase the log of running pipelines.

rm -rf pipelight/logs
# pkill pipelight

rm -rf pipelight/logs
# pkill pipelight

Which erases every log files.

The pipeline logs ​

Pipeline status (global status) ​

Verbosity levels ​

Error level (default) ​

Warn level (-v) ​

Info level (-vv) ​

Debug level (-vvv) ​

Trace level (-vvvv) ​

Raw logs (Json) ​

Other commands ​