SBT plugin – Introduction

To create a SBT plugin, I am using SBT 1.2.6 and Scala 2.12.x. My consumer project is using SBT 1.2.6 and Scala 2.11.12.

When you build a plugin, the requirements are:

• SBT version of the plugin needs to be the same as the SBT version of your project.
• SBT version of the plugin needs to match the Scala version of the plugin.
  • SBT 1.2.6 was built with Scala 2.12.x so that’s what I need to use for my plugin.
  • SBT 0.13.17 is built with Scala 2.10.x.

I have not seen many differences between SBT 0.13.x and SBT 1.2.x . I added a part with the differences I noticed.

If you want to know more, visit the documentation for SBT.

Keywords to know and that I am going to use

Definitions

• Module: This is a library. In build.sbt, you create it by doing: "org" % "name" % "revision".
  • ModuleID is the type in the plugin source code.
• Task: This is something you can call by doing sbt myTask.
• Setting: This is something you can set a value to in build.sbt. ( mySetting := ??? )
• Step: For task and setting, I am using the word step to describe both. In the SBT documentation it is referred as Key, but I think it is confusing.
• Scope: For SBT 0.13.17, this is global, project or build. When you use it in sbt console:
  • Build: {.}
  • Global: *
  • Project: None
• Configuration: This is when you use in in your build.sbt, like Test or Compile. When you use it in sbt console:
  •  You will have Scope/Configuration:Step.
  • In the code, they are starting by an uppercase letter.
• For 1.2.6:
  • You have Global, ThisBuild, Compile, Test, etc… You do not have weird symbol like * anymore.
  • And also it seems that / and : are not as distinct anymore in this version

Example for SBT 0.13.17

Below, you will find examples of the conversion of steps from the sbt console to the plugin source code terminology.

• myStep :
  • Scope: Project
  • Configuration: Root
  • Step: myStep
• */*:myStep :
  • Scope: Project
  • Configuration: Root
  • Step: myStep
• {.}/Compile:compile :
  • Scope: Build
  • Configuration: Compile
  • Step: compile
• */Test:update :
  • Scope: Global
  • Configuration: Test
  • Step: update

Example for SBT 1.2.6

The structure is the same but the * does not exist anymore. I’ll copy paste what I said above:

For 1.2.6:

• You have Global, ThisBuild, Compile, Test, etc… You do not have weird symbol like * anymore.
• And also it seems that / and : are not as distinct anymore in this version.

Difference between SBT 0.13.x and 1.2.x

dependencyOverrides

In 0.13.x it is a Set but in 1.2.x it is a Seq.

The method configuration() take a string instead of a ConfReg

This change is minor but it caused me issues.

In 0.13.x , the signature is:

def configuration(s: String): Option[ConfigurationReport]

But in 1.2.x , the signature is:

def configuration(s: ConfigRef): Option[ConfigurationReport]

But do not panic, ConfigRef is a class with this definition:

class ConfigRef(name: String)

So you can just use the configuration.name instead of configuration.

Useful commands

The following commands will help you debug and understand what is going on during each task process.

I would advise you to look at some commands like update, compile, sbtVersion, scalaVersion, etc…

And in different Configuration like Compile, Test, etc…

To inspect steps in SBT

Open the SBT console by typing sbt in your terminal.

Then, you can use the inspect actual command.

For instance, you can do:

> inspect actual */*:sbtVersion

[info] Setting: java.lang.String = 0.13.17
[info] Description:
[info] 	Provides the version of sbt.  This setting should be not be modified.
[info] Provided by:
[info] 	*/*:sbtVersion
[info] Defined at:
[info] 	(sbt.Defaults) Defaults.scala:136
[info] Dependencies:
[info] 	*/*:appConfiguration
[info] Reverse dependencies:
[info] 	*/*:sbtResolver
[info] 	*/*:sbtDependency
[info] 	*/*:pluginCrossBuild::sbtVersion
[info] 	*/*:sbtBinaryVersion
[info] 	*:sbtVersion
[info] Delegates:
[info] 	*/*:sbtVersion
[info] Related:
[info] 	*/*:pluginCrossBuild::sbtVersion
[info] 	*:sbtVersion

It is very useful to know which steps depend on which steps.

For instance, here we see that sbtVersion depends on */*:appConfiguration. There are 4 steps which need it, you can see that in the Reverse dependencies.

With 1.2.6, you do not have the * anymore, instead there are new keywords for all the scopes and configurations. It also seems that : is replaced by /.

Get the full dependencies of a step

You can use inspect tree [step].

> inspect tree *:libraryDependencies
[info] *:libraryDependencies = List(org.scala-lang:scala-library:2.11.12)
[info] +-*/*:autoScalaLibrary = true
[info] +-*/*:managedScalaInstance = true
[info] +-*/*:sbtPlugin = false
[info] +-*/*:scalaHome = None
[info] +-*/*:scalaOrganization = org.scala-lang
[info] +-*:scalaVersion = 2.11.12
[info]

Feel free to try with compile or update, the log is much bigger.

With 1.2.6, you do not have the * anymore, instead there are new keywords for all the scopes and configurations. It also seems that : is replaced by /.

Example of step dependency tree

By running the previous commands several times, I was able to create this dependency tree. I was curious to know how update get its list of dependencies coming from libraryDependency.

Get the output of a step

To get the output of a step you can use the show command.

For instance:

> show scalaVersion
[info] 2.11.12

You will see the output of the step. Feel free to try with update and try with different Configuration like Compile vs *.

Get the detailed execution

Using the command last right after the execution of a task will give you a detailed log of what just happened. You can also do it for each individual step. The SBT documentation will have more detailed information.

Project Structure: How to start your plugin?

Finally!

Here is the structure of my project folder:

$> tree -L 3 -I target .

.
├── README.md
├── build.sbt
├── project.sbt
├── publish.sbt
├── project
│    ├── build.properties
│    └── plugins.sbt
└── src
     └── main
           └── scala

Here is a nice screenshot of what I get (if the weird ASCII characters does not work above) :

Below, I will go through each file:

README.md

Readme.md is describing your project and this is written in markdown syntax. If you have never seen a Readme before, you can look at examples on Github: Search for README.md on github.

build.sbt

crossSbtVersions := Seq("0.13.17")

This file is pretty much the same thing as the build.sbt for any other SBT project. It contains the libraries you need inside your SBT plugin. One particular thing for an SBT plugin is crossSbtVersions which allow you to compile for several SBT versions. There are plenty of examples of how to do that on the internet and we won’t go over that here. You can begin by taking a look at this project: sbt dependency graph on Github for a good starting example.

project.sbt

This is where you define the version and names of your SBT plugin:

sbtPlugin := true

organization := "com.myorg"

name := "mySuperPlugin"
version := "1.2.3"

sbtPlugin := true is telling SBT that this is an SBT plugin. Important.

organization, name and version is for when you are going to load this SBT plugin inside your consumer project:

"organization" % "name" % "version"

publish.sbt

This file is everything related to how to deploy your SBT plugin:

publishMavenStyle := true
publishTo := Some("name of my artifactory" at "https://artifactory.myorg.com/artifactory/")

publishArtifact in Test := false

pomIncludeRepository := { _ => false }

You can just copy paste the above code and change the destination. I have read that people have issues with organization from the project.sbt file when it comes to publish: if you are only on github you might have to use com.github for organization.

Also, while you are debugging you can simply use sbt publishLocal which will publish the SBT plugin locally, and can be used by other projects.

project/build.properties

This file is setting the SBT version for this SBT plugin.

sbt.version=1.2.6

In my case, I am using 1.2.6  ( which is the latest at the time I am writing this post ) but you can set it to 0.13.17 ( which is the most commonly used stable previous major version ) .

project/plugin.sbt

This file is where you would load SBT plugin for your SBT plugin. Yeah inception style.

src/main/scala/com/myorg/MyMainPluginFile.scala

This is where your main file will be. You can go to the next part of this post to read more about its content.

SBT Plugin components and techniques

Major imports for SBT plugin

Most of your code will use the imports:

import sbt._

sbt is the core of everything.

To get a step which already exists in sbt, you have to use Keys. For instance: Keys.compile or Keys.update.

You can do import sbt.Keys._ , if you do not want to have to type it every time. But your code might be more readable if you do use Keys. every time you are using a pre-defined step.

The main class: AutoPlugin

The main class is the AutoPlugin:

object MyPluginMainClass extends AutoPlugin

This is where most of the architecture of your SBT plugin will go.

Log and Error and Exception

Error / Exception

To throw an error within the compile step you use:

sys.error

This method will throw something like that in the console:

java.lang.RuntimeException: This is an error message
	at scala.sys.package$.error(package.scala:27)
	[...more stack trace here...]
[error] ([RUNNING TASK NAME]) This is an error message
[error] Total time: [TIME] s, completed [DATE]

Logger

To fetch the logger within your step:

myTask := {
    val log = sbt.Keys.streams.value.log
    log.debug("debug message")
    log.info("info message")
    log.warn("warning message")
    log.error("error message")
}

With the above code, sbt myTask will yield:

[info] info message
[error] error message
[success] Total time: [TIME] s, completed [DATE]

to see all the messages, you need to do:

$> sbt 'set logLevel := Level.Debug' myTask
...
...some more log...
...
[debug] debug message
[info] info message
[warn] warning message
[error] error message
[success] Total time: [TIME] s, completed [DATE]

Notes

Printing error message with log.error does not trigger a failed task.

Since streams is a task, you cannot use it within a setting. You will get an error like: A setting cannot depend on a task.

If you look at how to log in settingkey in sbt on Stackoverflow you will have two options:

• Wrap your setting inside a task.
• Use a hard-coded ConsoleLogger.

If you want to learn more about the task in a setting, there is a part about this below.

Enable your SBT plugin

In you main object, the one extending AutoPlugin, you need to override the trigger method:

override def trigger = allRequirements

SBT plugin Task/Setting

Below are descriptions on how to use tasks and settings.

Settings

Settings are things that are being set to a value in the build.sbt, for example libraryDependencies.

It is evaluated once when sbt console start.

Find an example of a declared setting below:

val mySetting = settingKey[TypeOfMySetting]("The description of my setting")

Then the consumers of your SBT plugin will set the setting, in their build.sbt:

mySettingName := ???

To make sure that your setting is set to a value by the consumer of your SBT plugin, here is what you can do:

mySetting := {
  val mySettingValue =mySetting.??(undefinedKeyError(mySetting.key)).value
  mySettingValue
},

The key is with this custom method:

private def undefinedKeyError[A](key: AttributeKey[A]): A = {
  sys.error(s"Please declare a value for the `${key.label}` key. " +
    s"Description: '${key.description.getOrElse("A required key")}'"
  )
}

I found this method at Tapad/sbt-tweeter on Github which is also a great plugin to look at for examples. I found this repository through an exercise in complex sbt plugin development.

Tasks

Tasks are actions, like compile or update. They are evaluated once and only once per task call.

They are declared like this:

val myTask = taskKey[TypeOfMyTask]("The description of my task")

Then the consumer of your SBT plugin will use the task in the SBT console:

$> sbt
...starting the sbt console...

> myTask
...execute your task...
[success] Total time: [TIME] s, completed [DATE]

or all at once with sbt myTask inside your bash terminal.

Declare your steps

Inside your AutoPlugin class, you need to have an object:

object autoImport {
   val mySetting = settingKey[MyType]("description")
   val myTask = taskKey[MyOtherType]("description")
}

Once you have this sub-object inside your version of AutoPlugin you need to import it:

import autoImport._

This is where all the public steps will go. Those steps (task and setting) will be exposed by your SBT plugin.

If you do not want to expose it, you can leave it in the core object as private.

Usually, SBT plugins have another file name MyPluginKeys which looks like that:

import sbt.{settingKey, taskKey}

object SafetyPluginKeys {
  val mySetting = settingKey[MyType]("description")
  val myTask = taskKey[MyOtherType]("description")
  // more settings and tasks
}

Then your autoImport will become:

object autoImport {
   val mySetting = SafetyPluginKeys.mySetting
   val myTask = SafetyPluginKeys.myTask
}
import autoImport._

for the exposed one and just

import SafetyPluginKeys._

for the others.

Implement your steps

There are 3 levels of steps in an SBT plugin:

Project

You have to override:

override def projectSettings: Seq[Def.Setting[_]]

To be able to set a step in the project scope:

override def projectSettings: Seq[Def.Setting[_]] = { Seq[Def.Setting[_]](
    mySetting := {
      /* Create my value here */
    }
) }

When you use the command inspect actual , this correspond to looking at mySettingName in the SBT console.

Global

You have to override:

override def globalSettings: Seq[Def.Setting[_]]

To be able to set a step in the global scope:

override def globalSettings: Seq[Def.Setting[_]] = { Seq[Def.Setting[_]](
    mySetting := {
      /* Create my value here */
    }
) }

When you use the command inspect actual , this correspond to looking at */mySettingName in the SBT console for SBT 0.13.17.

Build

You have to override:

override def buildSettings: Seq[Def.Setting[_]]

To be able to set a step in the build scope:

override def buildSettings: Seq[Def.Setting[_]] = { Seq[Def.Setting[_]](
    mySetting := {
      /* Create my value here */
    },
    myTask := {
      /* ... */
    }
) }

When you use the command inspect actual , this correspond to looking at {.}/mySettingName in the SBT console for SBT 0.13.17.

How to use Configuration ?

In any of those three methods: projectSettings, globalSettings, buildSettings.

To add a configuration, you need to use the method in:

mySettingName in Compile := {
    /* Create my value here */
}

With this code, you are overriding the Compile:mySettingName which would be Compile/mySettingName in the SBT console when you are testing your plugin.

Also, in can be used with other step as input, not only configuration :

• For instance, you can have mySetting in update, which mean that this will override mySetting when called by update.
• You can also nest mySetting in update in Compile. Which in this case, makes it really confusing and I would advise to use .in(...) instead of the infix method.

This is true for sbt 0.13.17, for sbt 1.2.6, you have to use / instead of in. You can read more at “how to override the right task in sbt plugin” on Stackoverflow.

How to reuse my task code ?

If you have a complex step code and want to reuse it for let’s say Test and Compile, here is how you have to do it.

For Setting

For a setting, you will have to write a method like this:

private def mySettingMethod(): Def.Initialize[MySettingType] = {
  Def.settingDyn {
    // fetch dependencies
    // Remember, you can not use task inside a setting
    val dep1 = thisOtherSetting.value
    
    Def.setting {
      // the code of my setting
      theReturnValueOfMySettingOfTypeMySettingType
    }
  }
}

As you see, you will have to use Def.settingDyn to start up the context of this setting. And then, you will have to use Def.setting to write the code of this setting.

Now that you have this method, you can use it in several places:

override def projectSettings: Seq[Def.Setting[_]] = {
  Seq[Def.Setting[_]](
    // other steps ( task and setting )

    mySetting := mySettingMethod().value,
    mySetting in Compile := mySettingMethod().value,
    mySetting in Test := mySettingMethod().value,

    // other steps ( task and setting )
  )
}

Be careful not to forget to call the .value of the method.

If you call .value at the end of your method, you will get an error:

`value` can only be used within a task or setting macro, such as :=, +=, ++=, Def.task, or Def.setting.

So you have to create the context in your method and then call .value where you want it to be evaluated.

For Task

For a task, you will have to write a method like this:

private def myTaskMethod(): Def.Initialize[Task[MyTaskType]] = {
  Def.taskDyn {
    // fetch dependencies
    val dep1 = thisOtherSetting.value
    val dep2 = thisOtherTask.value
    
    Def.task {
      // the code of my task
      theReturnValueOfMyTaskOfTypeMyTaskType
    }
  }
}

As you see, you will have to use Def.taskDyn to start up the context of this task. And then, you will have to use Def.task to write the code of this task.

Now that you have this method, you can use it in several places:

override def projectSettings: Seq[Def.Setting[_]] = {
  Seq[Def.Setting[_]](
    // other steps ( task and setting )

    myTask := myTaskMethod().value,
    myTask in Compile := myTaskMethod().value,
    myTask in Test := myTaskMethod().value,

    // other steps ( task and setting )
  )
}

Be careful not to forget to call the .value of the method.

If you call .value at the end of your method, you will get an error:

`value` can only be used within a task or setting macro, such as :=, +=, ++=, Def.task, or Def.setting.

So you have to create the context in your method and then call .value where you want it to be returned.

Note – with arguments

You can pass arguments to those methods, for instance you can pass the Configuration:

private def myMethod(configuration: Configuration): Def.Initialize[Task[MyTaskType]] = {
  Def.(task|setting)Dyn {
    val dep1 = thisOtherSetting.value in configuration

    Def.(task|setting) {
      theReturnValue
    }
  }
}

And then you can use it this way:

override def projectSettings: Seq[Def.Setting[_]] = {
  Seq[Def.Setting[_]](
    // other steps ( task and setting )
    myStep in Compile := myMethod(Compile).value,
    myStep in Test := myMethod(Test).value,

    // other steps ( task and setting )
  )
}

How to override a setting ?

Let’s say you want to modify the output of a setting, for instance always add a library to compile:libraryDependencies.

libraryDependencies in Compile := {
     val currentLibs = (libraryDependencies inCompile).value
     currentLibs ++ Seq("org" % "name" % "revision")
}

You see the method .value which allow you to get the result of a step ( setting or task).

Now you can use one of the commands in sbt that we saw in the first part of this article ; here we would want to use show.

So go to the sbt console in the project which use your SBT plugin and type show Compile/libraryDependencies. Then, you should see the new ModuleID, that you added.

How to run a step before another task ?

For this section and the next one, thanks to “sbt plugin run tasks before after an other task” on StackOverflow.

Let’s say you want to run myTask before the task Compile/update.

You would use dependsOn:

myTask := {
    /* Create my value here */
    println("test")
},
update in Compile := (compile in Compile).dependsOn(myTask)

Now, when you go to the sbt console in your consumer project, you will see "test" when you execute the step Compile/compile. You can also see when you do inspect actual Compile/compile that myTask show up as dependencies.

How to run a task after an other task ?

For this section, and the previous one, thanks to “sbt plugin run tasks before after an other task” on StackOverflow.

This is a bit more tricky since that is not really build-in.

So to do that in your SBT plugin, you need to override a task, make it run, then do your step and then return the result you previously fetch:

update := Def.taskDyn {
  val updateResult = update.value

  Def.task {
    val _ = myOtherTask.value

    updateResult
  }
}.value

• Def.taskDyn and Def.task are to create the structure of the task.
• .value is the same we saw before, it allow you to get the result of a step.
• Do not forget the .value at the end

When you execute update in the sbt console, you will be able to see the logs of myOtherTask after the update one.

Note

You can, before .value have .dependsOn so you can have a task running before and another task running after.

Why can’t I use a task inside a setting ?

You cannot use a task inside setting because:

• A setting is evaluated once when sbt start.
• A task is evaluated once and only once, per task.

For instance, if you are calling myTask which depends on mySetting and myParentTask:

1. mySetting would have been evaluated when sbt console have started.
2. myParentTask will be evaluated once, even if it is called by several other task. If, for instance, you have:
   1. myTask depends on taskA and taskB and both taskA and taskB depends on myParentTask
   2. Then, myParentTask will be evaluated only once and the result reused for taskA and taskB and myTask
   3. If you call myTask again, myParentTask will be re-evaluated.

How to use the dependency graph

I was trying to get the dependency graph to alter it. It was really complex but reading “sbt dependency graph” on Github helped me a lot. Particularly, thank you jrudolph for answering my question issues #169 about the step to fetch the full graph on dependencies.

Why can’t I publish to Artifactory?

If you are using sbt 1.2.6, sometime you will encounter :

java.net.ProtocolException: Unexpected status line: 0

When trying to sbt publish to artifactory.

To resolve this problem, you can add:

updateOptions := updateOptions.value.withGigahorse(false)

To your build.sbt.

I found the solution to this problem at this Github issue #3519. I also found a StackOverflow answer stating that it might be from a bug in the Http client.

Conclusion

Once your plugin is ready, you can follow these steps to make your plugin known.

Please leave me a comment if you have any questions, insights or advice.

I will also update this article as I learn more about SBT plugins.