Pipelight

Appearance

Triggers (Automation) ​

Here is the part you were waiting for! What is the point of writting pipelines if you still have to execute them by hand?

tl;dr

Triggers are a set of conditions that can instantly launch multiple pipelines when they are met.

Prerequesits ​

In order to enable pipelight triggers these commands have to be executed somewhere inside your project directory.

Triggers are opt-in

The configuration file was initially polled to enable required triggers. But this has raised crucial concurrency issues. Until there is a fix, this feature is disabled and ** triggers have to be explicitly enabled from the command line**.

Enable git-hooks (Optional) ​

Most of triggers only work inside a Git repository. Be sure to initialize a repo if you want to take advantage of them all.

sh
git init
git init

To enable git triggers (pipelight managed git hooks) on a fresh directory run:

sh
pipelight enable git-hooks
pipelight enable git-hooks

DANGER

For now, this operation overwrites the .git/hooks folder. Be sure to move your manually defined hooks elsewhere before enabling pipelight hooks.

Disable them with:

sh
pipelight disable git-hooks
pipelight disable git-hooks

Enable file watcher (Optional) ​

An instance of pipelight runs in the background and listens to filesystem. events.

Computiong ressources consumption

The listener remains easy on the os and consumes very poor ressources by using once again the kernel modules through rust most used crates.

sh
pipelight enable watcher
pipelight enable watcher

Disable it with:

sh
pipelight disable watcher
pipelight disable watcher

Define pipeline triggers ​

Make a combination of branches and actions for which to trigger the pipeline.

When triggers are added to a pipeline, the pipeline is not triggered until triggering requirements are met. Which means you need to checkout to the allowed branches or tags, and execute the allowed actions for the pipeline to run.

(debug): When verbosity is increased, the CLI tells you what to do if requirements are not met.

disable them with:

sh
pipelight disable git-hooks.
pipelight disable git-hooks.

Make a combination of branches and actions for which to trigger the pipeline.

ts
pipeline.triggers =
  {
    branches: ["main"],
    actions: ["pre-push"],
  },
];
pipeline.triggers =
  {
    branches: ["main"],
    actions: ["pre-push"],
  },
];
ts
type Trigger = TriggerBranch | TriggerTag;

type TriggerBranch = { branches?: string[]; actions?: Action[] };
type TriggerTag = { tags?: string[]; actions?: Action[] };
type Trigger = TriggerBranch | TriggerTag;

type TriggerBranch = { branches?: string[]; actions?: Action[] };
type TriggerTag = { tags?: string[]; actions?: Action[] };

Then, add triggers to your pipeline definition.

ts
//pipelight.ts
pipelines: [
  {
    name: "test",
    steps: [
      {
        name: "build",
        commands: ["yarn install", "yarn build"],
      },
    ],
    triggers: [
      {
        branches: ["main"],
        actions: ["pre-push"],
      },
    ],
  },
];
//pipelight.ts
pipelines: [
  {
    name: "test",
    steps: [
      {
        name: "build",
        commands: ["yarn install", "yarn build"],
      },
    ],
    triggers: [
      {
        branches: ["main"],
        actions: ["pre-push"],
      },
    ],
  },
];

Triggers behavior ​

Before we dive aby deeper into the triggers.

WARNING

Heavy workloads should be trigger in detach mode with:

ts
pipeline: {
  options: {
    attach: false;
  }
}
pipeline: {
  options: {
    attach: false;
  }
}

To stick to the git-hooks trivial usage, pipelines are triggerd attached to the standard output. Which means your git actions wait for the pipelines to be executed.

To prevent waiting forever when triggering heavy workloads via a git action, you can set the pipeline to be executed detached from the standard output.

ts
const my_pipeline = {
  name: "always_detached_when_triggered_by_git",
  steps: [],
  options: {
    attach: false,
  },
};
const my_pipeline = {
  name: "always_detached_when_triggered_by_git",
  steps: [],
  options: {
    attach: false,
  },
};

Note that this flag only influence the behavior of the pipelight trigger command, and does nothing to the pipelight run.

You can set the default log level that the pipeline outputs with the log_level property.

ts
pipeline: {
  options: {
    attach: false;
    log_level: "warn",
  }
}
pipeline: {
  options: {
    attach: false;
    log_level: "warn",
  }
}

Available levels are error, warn, info, debug and trace.

Git environment (optional) ​

Branch and Tags ​

Branches are your git project branches names (see: git branch). Tags are the commits you made with git tag -a "v0.8" (see: git tag).

Tags are the tag you add to the commits you want to release with git tag -a "v0.8" (see: git tag).

Branch and Tag combinations are enhanced by globbing pattern matching.

ts
triggers: [
  {
    branches: ["feature/*"],
    actions: ["pre-push"],
  },
  {
    tags: ["v*-dev"],
    actions: ["pre-commit"],
  },
];
triggers: [
  {
    branches: ["feature/*"],
    actions: ["pre-push"],
  },
  {
    tags: ["v*-dev"],
    actions: ["pre-commit"],
  },
];

Actions ​

Actions are named according to git-hooks names, plus special flags "manual","watch" and "blank".

ts
export enum Action {
  // mail hooks
  ApplypatchMsg = "applypatch-msg",
  PreApplypatch = "pre-applypatch",
  PostApplypatch = "post-applypatch",
  SendemailValidate = "sendemail-validate",
  // client hooks
  PreCommit = "pre-commit",
  PreMergeCommit = "pre-merge-commit",
  PrepareCommitMsg = "prepare-commit-msg",
  CommitMsg = "commit-msg",
  PostCommit = "post-commit",
  // other client hooks
  PreRebase = "pre-rebase",
  PostCheckout = "post-checkout",
  PostMerge = "post-merge",
  PrePush = "pre-push",
  PostRewrite = "post-rewrite",
  PreReceive = "pre-receive",
  PreAutoGc = "pre-auto-gc",
  FsmonitorWatchman = "fsmonitor-watchman",
  PostIndexChange = "past-index-change",
  // p4
  P4Changelist = "p4-changelist",
  P4PrepareChangelist = "p4-prepare-changelist",
  P4PostChangelist = "p4-post-changelist",
  P4PreSubmit = "p4-pre-submit",
  // server-side hooks
  PreReceive = "pre-receive",
  Update = "update",
  ProcReceive = "proc-receive",
  PostReceive = "post-receive",
  PostUpdate = "post-update",
  RefrenceTransaction = "reference-transaction",
  PushToCheckout = "push-to-checkout",
  // special flags
  Manual = "manual",
  Watch = "watch",
  Blank = "blank",
}
export enum Action {
  // mail hooks
  ApplypatchMsg = "applypatch-msg",
  PreApplypatch = "pre-applypatch",
  PostApplypatch = "post-applypatch",
  SendemailValidate = "sendemail-validate",
  // client hooks
  PreCommit = "pre-commit",
  PreMergeCommit = "pre-merge-commit",
  PrepareCommitMsg = "prepare-commit-msg",
  CommitMsg = "commit-msg",
  PostCommit = "post-commit",
  // other client hooks
  PreRebase = "pre-rebase",
  PostCheckout = "post-checkout",
  PostMerge = "post-merge",
  PrePush = "pre-push",
  PostRewrite = "post-rewrite",
  PreReceive = "pre-receive",
  PreAutoGc = "pre-auto-gc",
  FsmonitorWatchman = "fsmonitor-watchman",
  PostIndexChange = "past-index-change",
  // p4
  P4Changelist = "p4-changelist",
  P4PrepareChangelist = "p4-prepare-changelist",
  P4PostChangelist = "p4-post-changelist",
  P4PreSubmit = "p4-pre-submit",
  // server-side hooks
  PreReceive = "pre-receive",
  Update = "update",
  ProcReceive = "proc-receive",
  PostReceive = "post-receive",
  PostUpdate = "post-update",
  RefrenceTransaction = "reference-transaction",
  PushToCheckout = "push-to-checkout",
  // special flags
  Manual = "manual",
  Watch = "watch",
  Blank = "blank",
}

Git actions (Git-hooks) ​

Actions are named according to git-hooks names, plus special flags like blank,manual and watch.

Special actions ​

On file change (Watch Flag) ​

ts
actions: ["watch"];
actions: ["watch"];

Trigger pipelines on file change. Whether a file is created, deleted or modified the pipeline is triggered.

You can ignore folders or files by declaring them inside the .pipelight_ignore hidden file which stick to the .gitignore file specifications.

Security (Manual Flag) ​

ts
actions: ["manual"];
actions: ["manual"];

If you want to manually run a pipeline that has non-empty triggers, with the command pipelight run you need to add the special flag manual to the pipeline trigger's actions. This avoids unintentionnal manual triggering aspecialy on critical production branches.

Client-Server synchronisation (Blank Flag) ​

ts
actions: ["blank"];
actions: ["blank"];

When you have pipelight installed client and server side, and you use detahed pipeline triggering with git.

A push to the (git)server triggers both client and server side pipelines nearly simultaneously.

client

on pre-push

server

on update

What if you want to trigger a server side pipeline only once a client side pipeline has resolved whitout having to bring it to the foreground?

A workaround is to send an ssh command to the server at some point in your pipeline. pipelight run <pipeline_name> --flag blank or pipelight trigger --flag blank to trigger the server side pipeline.

And add the required trigger to the pipeline to be executed server-side.

ts
// pipelight.ts
server_pipeline.triggers.push({
  action: ["blank"],
});
// pipelight.ts
server_pipeline.triggers.push({
  action: ["blank"],
});

Emit the trigger signal from the client-side pipeline. Whether it be from a fallback or from a regular pipeline step.

ts
// pipelight.ts
pipeline.on_success = {
  name: "sync",
  commands: `ssh ${host} -C "pipelight run server_side_pipeline --flag blank"`,
};
// pipelight.ts
pipeline.on_success = {
  name: "sync",
  commands: `ssh ${host} -C "pipelight run server_side_pipeline --flag blank"`,
};

and voilà, you have synced your pipelines.

client

on pre-push

trigger --flag blank

ssh

server

on blank

Forced flags (Manually set trigger action) ​

Simulate a specific action to trigger associated pipelines.

sh
pipelight trigger <action>
pipelight trigger <action>

Or trigger a pipeline by simulating the appropriate action.

sh
pipelight run --flag <action>
pipelight run --flag <action>

You can use it for debugging purpose or simply as a way to create unconventionnal pipelines.