{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": { "ExecuteTime": { "end_time": "2018-01-10T22:15:31.021568Z", "start_time": "2018-01-10T22:15:31.010972Z" }, "init_cell": true }, "outputs": [], "source": [ "from traitlets.config.manager import BaseJSONConfigManager\n", "from os.path import expanduser\n", "home = expanduser(\"~\")\n", "path = home+\"/.jupyter/nbconfig\"\n", "cm = BaseJSONConfigManager(config_dir=path)\n", "cm.update(\"livereveal\", {\n", " \"theme\": \"solarized\",\n", " \"transition\": \"zoom\",\n", " \"start_slideshow_at\": \"selected\"\n", "});\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# TeamCity\n", "\n", "
\n", "
\n", "\n", "
\n", "
\n", "\n", "\n", "\n", "

\n", " Alexis Jeandet :
\n", " Github:jeandet |\n", " Gitter:jeandet |\n", " IRC@freenode:jeandet\n", "
\n", "

\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# TC quick overview\n", "\n", "* A powerful distributed Continuous Integration tool\n", "* Rich plugin ecosystem\n", "* Really customizable system\n", "* Not so complex...\n", "* Updates so easy (Compared to redmine...)\n", "* Mostly written in Java\n", "* Closed source :(\n", "* Limited to 100 build cfg & 3 agents " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# TC Plugins (335 on [JetBrains website](https://plugins.jetbrains.com/teamcity) )\n", "\n", "* Web Hooks plugin \n", "* AWS S3 Artifact Storage \n", "* Google Cloud Agents \n", "* Docker Cloud \n", "* Debian Package Server \n", "* Slack Build Notifier\n", "\n", "\n", "** Installation as simple as uploading a zip file! **\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "ExecuteTime": { "end_time": "2018-01-02T20:14:04.757140Z", "start_time": "2018-01-02T20:14:04.749828Z" }, "slideshow": { "slide_type": "slide" } }, "source": [ "# Vocabulary\n", "\n", "* **Agent**: where stuff is done" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* **Build step**: the smallest build task subdivision ( [example](https://hephaistos.lpp.polytechnique.fr/teamcity/admin/editBuildRunners.html?id=buildType:SciQLop_UnitTests) ) " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* **Build config**: a collection of build steps ( [example](https://hephaistos.lpp.polytechnique.fr/teamcity/admin/editBuild.html?id=buildType:SciQLop_UnitTests) )" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* **VCS root**: a mostly a VCS URL + some options ( [example](https://hephaistos.lpp.polytechnique.fr/teamcity/admin/editVcsRoot.html?action=editVcsRoot&vcsRootId=SciQLop_HttpsHephaistosLppPolytechniqueFrRhodecodeHgRepositoriesLppSciQLOPReposS&editingScope=buildType%3ASciQLop_UnitTests&cameFromUrl=%2Fteamcity%2Fadmin%2FeditBuildTypeVcsRoots.html%3Finit%3D1%26id%3DbuildType%253ASciQLop_UnitTests&cameFromTitle=Edit%20Build%20Configuration) )" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* **Project**: a collection of build configs + VCS + settings ( [example](https://hephaistos.lpp.polytechnique.fr/teamcity/admin/editProject.html?projectId=SciQLop) )" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "# Vocabulary\n", "\n", "* **Build artifact**: a file or a folder produced during the build ( [example](https://hephaistos.lpp.polytechnique.fr/teamcity/overview.html) )" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "\n", "\n", "\n", "\n", "\n", "* **Meta-Runners**: a build config that has been extracted to make a template ( [example](https://hephaistos.lpp.polytechnique.fr/teamcity/admin/editProject.html?projectId=_Root&tab=metaRunner) )" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# LPP goals (What we want)\n", "\n", "\n", "* Being able to build on different OS" ] }, { "cell_type": "markdown", "metadata": { "ExecuteTime": { "end_time": "2018-01-08T19:36:05.070215Z", "start_time": "2018-01-08T19:36:05.066706Z" }, "slideshow": { "slide_type": "fragment" } }, "source": [ "* Build inside a clean environment" ] }, { "cell_type": "markdown", "metadata": { "ExecuteTime": { "end_time": "2018-01-08T19:36:05.399625Z", "start_time": "2018-01-08T19:36:05.394453Z" }, "slideshow": { "slide_type": "fragment" } }, "source": [ "* Produce compatibility matrices" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* Being faster than Travis" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "# LPP goals (What we want)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* Customize workflow" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* Give users maximum freedom" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# LPP Setup\n", "
\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Available build agents\n", "\n", "* Based on [Docker cloud plugin](https://plugins.jetbrains.com/plugin/9306-docker-cloud)\n", "* Virtually anything that runs either inside **Docker** or **VirtualBox**(Vagrant)\n", "* Spawn on demand\n", "* For now **Windows**, **Linux**(Fedora, Ubuntu), **FreeBSD** and **Mac OSX** 10.12.2(Siera)\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# FreeBSD & Mac \n", "\n", "[meson@lpp](https://hephaistos.lpp.polytechnique.fr/teamcity/viewLog.html?buildId=6388&tab=buildLog&buildTypeId=mesonbuild_Build_2&logTab=tree&filter=all) [meson@travis](https://travis-ci.org/mesonbuild/meson/builds/325859215?utm_source=github_status&utm_medium=notification)\n", "\n", "[OSX agent](https://github.com/jeandet/teamcity-docker-agent-osx) [FreeBSD agent](https://github.com/jeandet/teamcity-docker-agent-freebsd)\n", "\n", "* Vagrant inside Docker\n", "* Few txt files produces several GB machines\n", "* Easy to deploy\n", "* Hard to debug\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# I want to choose where I get built!\n", "\n", "**Team city introduce the notion of build requirements.**\n", "\n", "- Agents expose [parameters](https://hephaistos.lpp.polytechnique.fr/teamcity/agentDetails.html?agentTypeId=72&tab=agentParameters)(<=> properties). ( [sources](https://github.com/jeandet/teamcity-docker-complete-agent/blob/master/Dockerfile#L36) )\n", "\n", "- For each build config, ones can declare some [requirements](https://hephaistos.lpp.polytechnique.fr/teamcity/admin/editRequirements.html?id=buildType:Miniphare_Build) that the agent has to match to get spawned." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# What about reports?\n", "\n", "* As build artifact ( [LFR](https://hephaistos.lpp.polytechnique.fr/teamcity/viewLog.html?buildId=292&tab=artifacts&buildTypeId=LfrFlightSoftware_Build), [Hephaistos Web](https://hephaistos.lpp.polytechnique.fr/teamcity/project.html?projectId=HephaistosWeb&tab=projectOverview), [LibCDF](https://hephaistos.lpp.polytechnique.fr/teamcity/project.html?projectId=LibCDF&tab=projectOverview) )\n", "* As Mail\n", "* As slack message\n", "* As stdout values ( [GStreamer Ninja](https://hephaistos.lpp.polytechnique.fr/teamcity/admin/editRunType.html?id=buildType:mesonbuild_GStreamerNightly&runnerId=RUNNER_69&cameFromUrl=%2Fteamcity%2Fadmin%2FeditBuildRunners.html%3Fid%3DbuildType%253Amesonbuild_GStreamerNightly%26init%3D1&cameFromTitle=) )\n", "* As XML File ( [LibCDF GTest](https://hephaistos.lpp.polytechnique.fr/teamcity/admin/editBuildFeatures.html?id=buildType:LibCDF_Build) )" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Interactions with other tools\n", "\n", "* SonarQube ([example](https://sonarcloud.io/dashboard?id=sciqlop), [config](https://hephaistos.lpp.polytechnique.fr/teamcity/admin/editRunType.html?id=buildType:SciQLop_Build&runnerId=RUNNER_22&cameFromUrl=%2Fteamcity%2Fadmin%2FeditBuildRunners.html%3Fid%3DbuildType%253ASciQLop_Build%26init%3D1&cameFromTitle=))\n", "* Slack ([example](https://sciqlop.slack.com/messages/C57HU1KBJ/), [config](https://hephaistos.lpp.polytechnique.fr/teamcity/slacknotifications/index.html?projectId=SciQLop))\n", "* Github, Bitbucket, VS Team Services...\n", "* Rhodecode ([Miniphare PR](https://hephaistos.lpp.polytechnique.fr/rhodecode/GIT_REPOSITORIES/LPP/phare/miniphare/miniphare/pull-request/410), [config](https://hephaistos.lpp.polytechnique.fr/teamcity/admin/editRunType.html?id=buildType:Miniphare_Build&runnerId=RUNNER_82&cameFromUrl=%2Fteamcity%2Fadmin%2FeditBuildRunners.html%3Fid%3DbuildType%253AMiniphare_Build%26init%3D1&cameFromTitle=))\n", "* [VS IDE plugin](https://confluence.jetbrains.com/display/TCD10/Visual+Studio+Addin), [Eclipse plugin](https://confluence.jetbrains.com/display/TCD10/Eclipse+Plugin), [IntelliJ-based IDEs](https://blog.jetbrains.com/teamcity/2017/10/teamcity-integration-with-intellij-based-ides/)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# API?\n", "\n", "* Quite easy to [understand](https://confluence.jetbrains.com/display/TCD10/REST+API) ( [buildTypes](https://hephaistos.lpp.polytechnique.fr/teamcity/app/rest/buildTypes/), [projects](https://hephaistos.lpp.polytechnique.fr/teamcity/app/rest/projects/) )\n", "* Almost everything can be done through API" ] }, { "cell_type": "code", "execution_count": 38, "metadata": { "ExecuteTime": { "end_time": "2018-01-10T22:12:48.694242Z", "start_time": "2018-01-10T22:12:48.466252Z" }, "slideshow": { "slide_type": "slide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"id\": \"_Root\",\n", " \"name\": \"\",\n", " \"description\": \"Contains all other projects\",\n", " \"href\": \"/app/rest/projects/id:_Root\",\n", " \"webUrl\": \"https://hephaistos.lpp.polytechnique.fr/teamcity/project.html?projectId=_Root\"\n", "}\n" ] } ], "source": [ "import teamcity\n", "import json\n", "\n", "print(json.dumps(teamcity.tc.projects()[0].__dict__, indent=1))" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# What Next?\n", "\n", "* Improve Rhodecode integration\n", "* Boost agents startup time\n", "* Switch to Docker swarm\n", "* More Meta-Runners" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Questions?\n" ] } ], "metadata": { "celltoolbar": "Slideshow", "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.3" }, "toc": { "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": {}, "toc_section_display": true, "toc_window_display": false }, "varInspector": { "cols": { "lenName": 16, "lenType": 16, "lenVar": 40 }, "kernels_config": { "python": { "delete_cmd_postfix": "", "delete_cmd_prefix": "del ", "library": "var_list.py", "varRefreshCmd": "print(var_dic_list())" }, "r": { "delete_cmd_postfix": ") ", "delete_cmd_prefix": "rm(", "library": "var_list.r", "varRefreshCmd": "cat(var_dic_list()) " } }, "types_to_exclude": [ "module", "function", "builtin_function_or_method", "instance", "_Feature" ], "window_display": false } }, "nbformat": 4, "nbformat_minor": 2 }