Berlin, Germany
“We make Live, Push and Link — unique software and hardware for music creation and performance.”
ableton.com
Multi-platform C++ desktop application
Rendering a Qt Quick scene on the display
Available to developers as an iOS SDK
1440 source files are compiled into 88 libraries and 8 executables when building Live
Generate Your Projects
Integrated in the live repository
live/
│
├── modules/
│ └── build-system/ # git submodule
│
│
│
│
│
│
│
│
│
└── Live.gyp
live/
│
├── modules/
│ └── build-system/ # git submodule
│ │
│ ├── modules/
│ │ └── gyp/ # git submodule
│ │
│ └── scripts/
│ ├── configure.py # calls GYP
│ ├── build.py # calls MSBuild, Ninja or xcodebuild
│ └── run.py # calls C++ test runners or Qt-specific tools
│
└── Live.gyp
HelloWorld project
HelloWorld/
│
├── build-system/ # git submodule
│ └── scripts/
│ ├── configure.py
│ ├── build.py
│ └── run.py
│
├── src/
│ └── HelloWorld.cpp
│
├── test/
│ ├── catch.hpp # from github.com/philsquared/Catch
│ └── tst_HelloWorld.cpp
│
└── HelloWorld.gyp
HelloWorld.gyp
{
'targets': [
{
'target_name': 'HelloWorld',
'type': 'executable',
'sources': [
'src/HelloWorld.cpp',
],
},
{
'target_name': 'HelloWorldTest',
'type': 'executable',
'sources': [
'test/tst_HelloWorld.cpp',
],
},
],
}
HelloWorld.cpp
#include <iostream>
int main()
{
std::cout << "Hello World!" << std::endl;
return 0;
}
tst_HelloWorld.cpp
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
TEST_CASE("Hello World!")
{
CHECK(true);
}
Configure
$ python build-system/scripts/configure.py
Generating projects from HelloWorld.gyp (win, 64, dll)
HelloWorld/
│
├── build-system/
│
├── ide/
│ └── win_64_dll/
│ ├── HelloWorld.sln
│ ├── HelloWorld.vcxproj
│ └── HelloWorldTest.vcxproj
│
├── src/
├── test/
└── HelloWorld.gyp
Build
$ python build-system/scripts/build.py
Building All targets from ide\win_64_dll\HelloWorld.sln (Debug)
Microsoft (R) Build Engine version 12.0.40629.0
[Microsoft .NET Framework, version 4.0.30319.42000]
Copyright (C) Microsoft Corporation. All rights reserved.
HelloWorld.cpp
HelloWorld.vcxproj -> ..\..\output\win_64_dll\Debug\HelloWorld.exe
tst_HelloWorld.cpp
HelloWorldTest.vcxproj -> ..\..\output\win_64_dll\Debug\HelloWorldTest.exe
Built All targets from ide\win_64_dll\HelloWorld.sln (Debug)
Build
HelloWorld/
│
├── build-system/
├── ide/
│
├── output/
│ └── win_64_dll/
│ └── Debug/
│ ├── HelloWorld.exe
│ └── HelloWorldTest.exe
│
├── src/
├── test/
└── HelloWorld.gyp
Bonjour le monde!
$ ./output/win_64_dll/Debug/HelloWorld.exe
Hello World!
Run C++ tests
$ python build-system/scripts/run.py cpptest
Running output\win_64_dll\Debug\HelloWorldTest.exe
========================================================================
All tests passed (1 assertion in 1 test case)
1 PASSED TEST SUITE
* output\win_64_dll\Debug\HelloWorldTest.exe
0 FAILED TEST SUITES
SUCCESS
Compared to the HelloWorld project
Simply call: configure.py, build.py, run.py
Doesn't guarantee that they are easy to
use, maintain and extend
It could still be a mess
Based solely on Python's standard library
import argparse
import logging
import subprocess
Composed of several design principles
Make the scripts easier to use, maintain and extend
Can be applied to your own scripts
Counting entries in a given directory
import argparse
import os
arg_parser = argparse.ArgumentParser()
arg_parser.add_argument('--target-dir')
args = arg_parser.parse_args()
target_dir = args.target_dir
num_entries = len(os.listdir(target_dir))
print('There are {} entries in {}'.format(num_entries, target_dir))
$ python count-entries.py --target-dir HelloWorld/
There are 6 entries in HelloWorld/
Counting entries in a non-existing directory
$ python count-entries.py --target-dir Nope/
Traceback (most recent call last):
File "count-entries.py", line 11, in <module>
num_entries = len(os.listdir(target_dir))
WindowsError: [Error 3] The system cannot find the path specified:
'Nope/*.*'
Check the argument type as early as possible
import argparse
import os
+ def existing_dir(path):
+ if not os.path.isdir(path):
+ raise argparse.ArgumentTypeError('No such directory: ' + path)
+ return path
+
arg_parser = argparse.ArgumentParser()
arg_parser.add_argument('--target-dir', type=existing_dir)
args = arg_parser.parse_args()
target_dir = args.target_dir
num_entries = len(os.listdir(target_dir))
print('There are {} entries in {}'.format(num_entries, target_dir))
$ python count-entries-fail-early.py --target-dir HelloWorld/
There are 6 entries in HelloWorld/
Fail early, loud and clear
$ python count-entries-fail-early.py --target-dir Nope/
usage: count-entries-fail-early.py [-h] [--target-dir TARGET_DIR]
count-entries-fail-early.py: error: argument --target-dir: No such
directory: Nope/
Building with Ninja
$ python build-system/scripts/configure.py --ninja
Generating projects from HelloWorld.gyp (win, 64, dll, ninja)
$ python build-system/scripts/build.py --ninja
ninja: Entering directory `output\win_64_dll\Debug`
[1/4] CXX obj\src\HelloWorld.HelloWorld.obj
[2/4] LINK_EMBED HelloWorld.exe
[3/4] CXX obj\test\HelloWorldTest.tst_HelloWorld.obj
[4/4] LINK_EMBED HelloWorldTest.exe
$ ./output/win_64_dll/Debug/HelloWorld.exe
Hello World!
build-system.rc files
$ cat ~/.ableton/build-system.rc
{
'configure': [
'--ninja',
],
'build': [
'--ninja',
],
'run': [
],
}
1. ~/.ableton/build-system.rc (per user)
2. <Project>/build-system/build-system.rc (per project, .gitignored)
3. command line
Building with Ninja by default
$ python build-system/scripts/configure.py
Generating projects from HelloWorld.gyp (win, 64, dll, ninja)
$ python build-system/scripts/build.py
ninja: Entering directory `output\win_64_dll\Debug`
[1/4] CXX obj\src\HelloWorld.HelloWorld.obj
[2/4] LINK_EMBED HelloWorld.exe
[3/4] CXX obj\test\HelloWorldTest.tst_HelloWorld.obj
[4/4] LINK_EMBED HelloWorldTest.exe
build-system repository
build-system/
│
├── modules/
│ └── gyp/
│
└── scripts/
├── build_system/
│ ├── __init__.py
│ ├── arg_utils.py # existing_dir()
│ ├── constants.py # get_default_platform()
│ └── path_defaults.py # gyp_file_path()
│
├── configure.py
├── build.py
└── run.py
build-system repository
build-system/
│
├── modules/
│ └── gyp/
│
└── scripts/
├── build_system/
│ ├── __init__.py
│ ├── arg_utils.py # existing_dir()
│ ├── constants.py # get_default_platform()
│ └── path_defaults.py # gyp_file_path()
│
├── configure.py
├── build.py
├── run.py
└── simple-arg-parser.py
simple-arg-parser.py
import argparse
from build_system import constants
arg_parser = argparse.ArgumentParser()
arg_parser.add_argument('--platform', choices=['mac', 'win'],
default=constants.get_default_platform())
arg_parser.add_argument('--wordsize', choices=['32', '64'],
default='64')
arg_parser.add_argument('--linking', choices=['dll', 'static'],
default='dll')
print(arg_parser.parse_args())
$ python simple-arg-parser.py
Namespace(linking='dll', platform='win', wordsize='64')
Project specific argument
HelloWorld/
├── build-system/
│ └── scripts/
│ ├── configure.py
│ ├── build.py
│ ├── run.py
│ └── simple-arg-parser.py
├── src/
├── test/
└── HelloWorld.gyp
$ python build-system/scripts/simple-arg-parser.py --lang=es
usage: simple-arg-parser.py [-h] [--platform {mac,win}]
[--wordsize {32,64}]
[--linking {dll,static}]
simple-arg-parser.py: error: unrecognized arguments: --lang=es
Can we add it to build-system?
Adds maintenance cost
Leads to API breaking changes
Extending the script
import argparse
from build_system import constants, script_extensions
+ extension = script_extensions.get_script_extension('simple_arg_parser', {
+ 'extend_arg_parser_before': lambda parser: None,
+ 'extend_arg_parser_after': lambda parser: None,
+ })
+
arg_parser = argparse.ArgumentParser()
+ extension.extend_arg_parser_before(arg_parser)
+
arg_parser.add_argument('--platform', choices=['mac', 'win'],
default=constants.get_default_platform())
arg_parser.add_argument('--wordsize', choices=['32', '64'],
default='64')
arg_parser.add_argument('--linking', choices=['dll', 'static'],
default='dll')
+ extension.extend_arg_parser_after(arg_parser)
+
print(arg_parser.parse_args())
Using the extension mechanism in the project
HelloWorld/
├── build-system/
│ └── scripts/
│ ├── configure.py
│ ├── build.py
│ ├── run.py
│ └── simple-arg-parser.py
│
├── build_system_extensions/
│ ├── __init__.py
│ └── simple_arg_parser.py
│
├── src/
├── test/
└── HelloWorld.gyp
simple_arg_parser.py
def extend_arg_parser_after(arg_parser):
arg_parser.add_argument('--lang', default='en')
Using the extension mechanism in the project
HelloWorld/
├── build-system/
│ └── scripts/
│ ├── configure.py
│ ├── build.py
│ ├── run.py
│ └── simple-arg-parser.py
│
├── build_system_extensions/
│ ├── __init__.py
│ └── simple_arg_parser.py
│
├── src/
├── test/
└── HelloWorld.gyp
$ python build-system/scripts/simple-arg-parser.py --lang=es
Namespace(lang='es', linking='dll', platform='win', wordsize='64')
Extending the script
HelloWorld/
├── build-system/
│ └── scripts/
│ ├── build_system/
│ │ ├── __init__.py
│ │ ├── arg_utils.py # existing_dir()
│ │ ├── constants.py # get_default_platform()
│ │ ├── path_defaults.py # gyp_file_path()
│ │ └── script_extensions.py # get_script_extension()
│ │
│ ├── configure.py
│ ├── build.py
│ ├── run.py
│ └── simple-arg-parser.py
│
├── build_system_extensions/
│ ├── __init__.py
│ └── simple_arg_parser.py
│
├── src/
├── test/
└── HelloWorld.gyp
Extending the script
from build_system import script_extensions
extension = script_extensions.get_script_extension(
script_name='simple_arg_parser',
minimal_api={
'extend_arg_parser_before': lambda arg_parser: None,
'extend_arg_parser_after': lambda arg_parser: None,
})
Importing a Python module "manually"
import imp
def find_and_load_module(name, paths):
try:
found_module = imp.find_module(name, paths)
if found_module:
module_file, module_path, module_desc = found_module
try:
return imp.load_module(name, *found_module)
finally:
if module_file is not None:
module_file.close()
except ImportError:
return None
Creating the script extension
def get_script_extension(script_name, minimal_api):
script_extension = None
extensions_module = find_and_load_module(
'build_system_extensions', _all_parent_folders(os.path.curdir))
if extensions_module:
script_extension = find_and_load_module(
script_name, extensions_module.__path__)
if script_extension is None:
script_extension = object()
for attr in minimal_api:
if not hasattr(script_extension, attr):
setattr(script_extension, attr, minimal_api[attr])
return script_extension
Python is indeed the keystone
“Without Python, nobody can work on Live.” Myself, 2 seconds ago
Having some is good
Knowing and sharing them is better
Thank you for your attention
Questions?
@AbletonDev