13228537945

Committed 09 Feb 2025 06:34PM UTC coverage: 35.131%. Remained the same

Build # 13228537945

Build Type

Pull #644

github

Committed by

web-flow

Commit Message

Merge 8b3aeb3fa into 16f1fc1f8

Pull Request Pull Request #644: Ignore archived projects

Run Details

437 of 1188 branches covered (36.78%)

645 of 1836 relevant lines covered (35.13%)

3.48 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

0.0

/tools/migrate_branch_empty.py

#!/usr/bin/env python
#  -*- coding: utf-8 -*-
# License AGPLv3 (https://www.gnu.org/licenses/agpl-3.0-standalone.html)
"""
This script helps to create a new branch for a new Odoo version, catching the
required metafiles from another existing branch, and making the needed changes
on contents.

Installation
============

For using this utility, you need to install these dependencies:

* github3.py library for handling Github calls. To install it, use:
  `sudo pip install github3.py`.

Configuration
=============

You must have a file called oca.cfg on the same folder of the script for
storing credentials parameters. You can generate an skeleton config running
this script for a first time.

Usage
=====
oca-migrate-branch-empty [-h] [-p PROJECTS [PROJECTS ...]] [-e EMAIL]
                        [-t TARGET_ORG]
                        source target

positional arguments:
  source                Source branch (existing)
  target                Target branch (to create)

optional arguments:
  -h, --help            show this help message and exit
  -p PROJECTS [PROJECTS ...], --projects PROJECTS [PROJECTS ...]
                        List of specific projects to migrate
  -e EMAIL, --email EMAIL
                        Provides an email address used to commit on GitHub if
                        the one associated to the GitHub account is not public
  -t TARGET_ORG, --target-org TARGET_ORG
                        By default, the GitHub organization used is OCA. This
                        arg lets you provide an alternative organization

This script will perform the following operations for each project:

* Create an empty branch with 'target' as name. If it already exists, then
  the project is skipped.
* Catch these possible metafiles from source branch, replacing all references
  to it by the target branch:
  * .travis.yml
  * .gitignore
  * CONTRIBUTING.md
  * LICENSE
  * README.md
* Make target branch the default branch in the repository.

Known issues / Roadmap
======================

* Pin the migration issue when GitHub API allows it.
* Issue enumerating the module list contains a list to a Wiki page that should
  be formatted this way:
  https://github.com/OCA/maintainer-tools/wiki/Migration-to-version-{branch}
* Make the created branch protected (no support yet from github3 library).

Credits
=======

Contributors
------------

* Pedro M. Baeza <pedro.baeza@tecnativa.com>

Maintainer
----------

.. image:: https://odoo-community.org/logo.png
   :alt: Odoo Community Association
   :target: https://odoo-community.org

This module is maintained by the OCA.

OCA, or the Odoo Community Association, is a nonprofit organization whose
mission is to support the collaborative development of Odoo features and
promote its widespread use.

To contribute to this module, please visit https://odoo-community.org.
"""

from __future__ import print_function

import argparse
import re

from github3.exceptions import NotFoundError

from . import github_login, oca_projects
from .config import read_config

MANIFESTS = ("__openerp__.py", "__manifest__.py")


class BranchMigrator(object):
    def __init__(self, source, target, target_org=None, email=None):
        # Read config
        config = read_config()
        self.gh_token = config.get("GitHub", "token")
        # Connect to GitHub
        self.github = github_login.login()
        gh_user = self.github.me()
        if not gh_user.email and not email:
            raise Exception(
                "Email required to commit to github. Please provide one on "
                "the command line or make the one of your github profile "
                "public."
            )
        self.gh_credentials = {
            "name": gh_user.name or str(gh_user),
            "email": gh_user.email or email,
        }
        self.gh_source_branch = source
        self.gh_target_branch = target
        self.gh_org = target_org or "OCA"

    def _replace_content(self, repo, path, replace_list, gh_file=None):
        if not gh_file:
            # Re-read path for retrieving content
            gh_file = repo.file_contents(path, self.gh_source_branch)
        content = gh_file.decoded.decode("utf-8")
        for replace in replace_list:
            content = re.sub(replace[0], replace[1], content, flags=re.DOTALL)
        new_file_blob = repo.create_blob(content, encoding="utf-8")
        return {
            "path": path,
            "mode": "100644",
            "type": "blob",
            "sha": new_file_blob,
        }

    def _create_commit(self, repo, tree_data, message, use_sha=True):
        """Create a GitHub commit.
        :param repo: github3 repo reference
        :param tree_data: list with dictionary for the entries of the commit
        :param message: message to use in the commit
        :param use_sha: if False, the tree_data structure will be considered
        the full one, deleting the rest of the entries not listed in this one.
        """
        if not tree_data:
            return
        if use_sha:
            branch = repo.branch(self.gh_target_branch)
            tree_sha = branch.commit.commit.tree.sha
            parents = [branch.commit.sha]
        else:
            tree_sha = None
            parents = []
        tree = repo.create_tree(tree_data, tree_sha)
        commit = repo.create_commit(
            message=message,
            tree=tree.sha,
            parents=parents,
            author=self.gh_credentials,
            committer=self.gh_credentials,
        )
        return commit

    def _create_metafiles(self, repo, root_contents):
        """Create metafiles (README.md, .travis.yml...) pointing to the new
        branch.
        """
        tree_data = []
        source_string = self.gh_source_branch.replace(".", r"\.")
        target_string = self.gh_target_branch
        source_string_dash = self.gh_source_branch.replace(".", "-")
        target_string_dash = self.gh_target_branch.replace(".", "-")
        REPLACES = {
            "README.md": {
                None: [
                    (source_string, target_string),
                    (source_string_dash, target_string_dash),
                    (r"\[//]: # \(addons\).*\[//]: # \(end addons\)", ""),
                ],
            },
            ".travis.yml": {
                None: [
                    (source_string, target_string),
                    (source_string_dash, target_string_dash),
                    (
                        r"(?i)([^\n]+ODOO_REPO=['\"]ODOO[^\n]+)\n([^\n]+"
                        r"ODOO_REPO=['\"]oca\/ocb[^\n]+)",
                        r"\2\n\1",
                    ),
                ],
                "11.0": [
                    ("2.7", "3.5"),
                    (r"(?m)virtualenv:.*\n.*system_site_packages: true\n", ""),
                ],
                "12.0": [
                    (r"addons:\n", r'addons:\n  postgresql: "9.6"'),
                ],
            },
            ".gitignore": {
                None: [],
            },
            "CONTRIBUTING.md": {
                None: [],
            },
            "LICENSE": {
                None: [],
            },
        }
        for filename in REPLACES:
            if not root_contents.get(filename):
                continue
            replaces = []
            for version in REPLACES[filename]:
                if version and self.gh_target_branch != version:
                    continue
                replaces += REPLACES[filename][version]
            tree_data.append(self._replace_content(repo, filename, replaces))
        commit = self._create_commit(
            repo,
            tree_data,
            "[MIG] Add metafiles\n\n[skip ci]",
            use_sha=False,
        )
        return commit

    def _make_default_branch(self, repo):
        repo.edit(repo.name, default_branch=self.gh_target_branch)

    def _migrate_project(self, project):
        print("Migrating project %s/%s" % (self.gh_org, project))
        # Create new branch
        repo = self.github.repository(self.gh_org, project)
        try:
            repo.branch(self.gh_source_branch)
        except NotFoundError:
            print("Source branch non existing. Skipping...")
            return
        try:
            repo.branch(self.gh_target_branch)
        except NotFoundError:
            pass
        else:
            print("Branch already exists. Skipping...")
            return
        root_contents = repo.directory_contents(
            "",
            self.gh_source_branch,
            return_as=dict,
        )
        commit = self._create_metafiles(repo, root_contents)
        repo.create_ref(
            "refs/heads/%s" % self.gh_target_branch,
            commit.sha,
        )
        # TODO: GitHub is returning 404
        # self._make_default_branch(repo)

    def do_migration(self, projects=None):
        if not projects:
            projects = oca_projects.get_repositories()
        for project in projects:
            self._migrate_project(project)


def get_parser():
    parser = argparse.ArgumentParser(
        description="Migrate one OCA branch from one version to another, "
        "applying the needed transformations",
        add_help=True,
    )
    parser.add_argument("source", help="Source branch (existing)")
    parser.add_argument("target", help="Target branch (to create)")
    parser.add_argument(
        "-p",
        "--projects",
        dest="projects",
        nargs="+",
        default=[],
        help="List of specific projects to migrate",
    )
    parser.add_argument(
        "-e",
        "--email",
        dest="email",
        help=(
            "Provides an email address used to commit on GitHub if the one "
            "associated to the GitHub account is not public"
        ),
    )
    parser.add_argument(
        "-t",
        "--target-org",
        dest="target_org",
        help=(
            "By default, the GitHub organization used is OCA. This arg lets "
            "you provide an alternative organization"
        ),
    )
    return parser


def main():
    args = get_parser().parse_args()
    migrator = BranchMigrator(
        source=args.source,
        target=args.target,
        target_org=args.target_org,
        email=args.email,
    )
    migrator.do_migration(projects=args.projects)


if __name__ == "__main__":
    main()

1	#!/usr/bin/env python
2	# -- coding: utf-8 --
3	# License AGPLv3 (https://www.gnu.org/licenses/agpl-3.0-standalone.html)
4	"""
5	This script helps to create a new branch for a new Odoo version, catching the
6	required metafiles from another existing branch, and making the needed changes
7	on contents.
8
9	Installation
10	============
11
12	For using this utility, you need to install these dependencies:
13
14	* github3.py library for handling Github calls. To install it, use:
15	`sudo pip install github3.py`.
16
17	Configuration
18	=============
19
20	You must have a file called oca.cfg on the same folder of the script for
21	storing credentials parameters. You can generate an skeleton config running
22	this script for a first time.
23
24	Usage
25	=====
26	oca-migrate-branch-empty [-h] [-p PROJECTS [PROJECTS ...]] [-e EMAIL]
27	[-t TARGET_ORG]
28	source target
29
30	positional arguments:
31	source Source branch (existing)
32	target Target branch (to create)
33
34	optional arguments:
35	-h, --help show this help message and exit
36	-p PROJECTS [PROJECTS ...], --projects PROJECTS [PROJECTS ...]
37	List of specific projects to migrate
38	-e EMAIL, --email EMAIL
39	Provides an email address used to commit on GitHub if
40	the one associated to the GitHub account is not public
41	-t TARGET_ORG, --target-org TARGET_ORG
42	By default, the GitHub organization used is OCA. This
43	arg lets you provide an alternative organization
44
45	This script will perform the following operations for each project:
46
47	* Create an empty branch with 'target' as name. If it already exists, then
48	the project is skipped.
49	* Catch these possible metafiles from source branch, replacing all references
50	to it by the target branch:
51	* .travis.yml
52	* .gitignore
53	* CONTRIBUTING.md
54	* LICENSE
55	* README.md
56	* Make target branch the default branch in the repository.
57
58	Known issues / Roadmap
59	======================
60
61	* Pin the migration issue when GitHub API allows it.
62	* Issue enumerating the module list contains a list to a Wiki page that should
63	be formatted this way:
64	https://github.com/OCA/maintainer-tools/wiki/Migration-to-version-{branch}
65	* Make the created branch protected (no support yet from github3 library).
66
67	Credits
68	=======
69
70	Contributors
71	------------
72
73	* Pedro M. Baeza <pedro.baeza@tecnativa.com>
74
75	Maintainer
76	----------
77
78	.. image:: https://odoo-community.org/logo.png
79	:alt: Odoo Community Association
80	:target: https://odoo-community.org
81
82	This module is maintained by the OCA.
83
84	OCA, or the Odoo Community Association, is a nonprofit organization whose
85	mission is to support the collaborative development of Odoo features and
86	promote its widespread use.
87
88	To contribute to this module, please visit https://odoo-community.org.
89	"""
90
91	from __future__ import print_function	×
92
93	import argparse	×
94	import re	×
95
96	from github3.exceptions import NotFoundError	×
97
98	from . import github_login, oca_projects	×
99	from .config import read_config	×
100
101	MANIFESTS = ("__openerp__.py", "__manifest__.py")	×
102
103
104	class BranchMigrator(object):	×
105	def __init__(self, source, target, target_org=None, email=None):	×
106	# Read config
107	config = read_config()	×
108	self.gh_token = config.get("GitHub", "token")	×
109	# Connect to GitHub
110	self.github = github_login.login()	×
111	gh_user = self.github.me()	×
112	if not gh_user.email and not email:	×
113	raise Exception(	×
114	"Email required to commit to github. Please provide one on "
115	"the command line or make the one of your github profile "
116	"public."
117	)
118	self.gh_credentials = {	×
119	"name": gh_user.name or str(gh_user),
120	"email": gh_user.email or email,
121	}
122	self.gh_source_branch = source	×
123	self.gh_target_branch = target	×
124	self.gh_org = target_org or "OCA"	×
125
126	def _replace_content(self, repo, path, replace_list, gh_file=None):	×
127	if not gh_file:	×
128	# Re-read path for retrieving content
129	gh_file = repo.file_contents(path, self.gh_source_branch)	×
130	content = gh_file.decoded.decode("utf-8")	×
131	for replace in replace_list:	×
132	content = re.sub(replace[0], replace[1], content, flags=re.DOTALL)	×
133	new_file_blob = repo.create_blob(content, encoding="utf-8")	×
134	return {	×
135	"path": path,
136	"mode": "100644",
137	"type": "blob",
138	"sha": new_file_blob,
139	}
140
141	def _create_commit(self, repo, tree_data, message, use_sha=True):	×
142	"""Create a GitHub commit.
143	:param repo: github3 repo reference
144	:param tree_data: list with dictionary for the entries of the commit
145	:param message: message to use in the commit
146	:param use_sha: if False, the tree_data structure will be considered
147	the full one, deleting the rest of the entries not listed in this one.
148	"""
149	if not tree_data:	×
150	return	×
151	if use_sha:	×
152	branch = repo.branch(self.gh_target_branch)	×
153	tree_sha = branch.commit.commit.tree.sha	×
154	parents = [branch.commit.sha]	×
155	else:
156	tree_sha = None	×
157	parents = []	×
158	tree = repo.create_tree(tree_data, tree_sha)	×
159	commit = repo.create_commit(	×
160	message=message,
161	tree=tree.sha,
162	parents=parents,
163	author=self.gh_credentials,
164	committer=self.gh_credentials,
165	)
166	return commit	×
167
168	def _create_metafiles(self, repo, root_contents):	×
169	"""Create metafiles (README.md, .travis.yml...) pointing to the new
170	branch.
171	"""
172	tree_data = []	×
173	source_string = self.gh_source_branch.replace(".", r"\.")	×
174	target_string = self.gh_target_branch	×
175	source_string_dash = self.gh_source_branch.replace(".", "-")	×
176	target_string_dash = self.gh_target_branch.replace(".", "-")	×
177	REPLACES = {	×
178	"README.md": {
179	None: [
180	(source_string, target_string),
181	(source_string_dash, target_string_dash),
182	(r"\[//]: # \(addons\).*\[//]: # \(end addons\)", ""),
183	],
184	},
185	".travis.yml": {
186	None: [
187	(source_string, target_string),
188	(source_string_dash, target_string_dash),
189	(
190	r"(?i)([^\n]+ODOO_REPO=['\"]ODOO[^\n]+)\n([^\n]+"
191	r"ODOO_REPO=['\"]oca\/ocb[^\n]+)",
192	r"\2\n\1",
193	),
194	],
195	"11.0": [
196	("2.7", "3.5"),
197	(r"(?m)virtualenv:.\n.system_site_packages: true\n", ""),
198	],
199	"12.0": [
200	(r"addons:\n", r'addons:\n postgresql: "9.6"'),
201	],
202	},
203	".gitignore": {
204	None: [],
205	},
206	"CONTRIBUTING.md": {
207	None: [],
208	},
209	"LICENSE": {
210	None: [],
211	},
212	}
213	for filename in REPLACES:	×
214	if not root_contents.get(filename):	×
215	continue	×
216	replaces = []	×
217	for version in REPLACES[filename]:	×
218	if version and self.gh_target_branch != version:	×
219	continue	×
220	replaces += REPLACES[filename][version]	×
221	tree_data.append(self._replace_content(repo, filename, replaces))	×
222	commit = self._create_commit(	×
223	repo,
224	tree_data,
225	"[MIG] Add metafiles\n\n[skip ci]",
226	use_sha=False,
227	)
228	return commit	×
229
230	def _make_default_branch(self, repo):	×
231	repo.edit(repo.name, default_branch=self.gh_target_branch)	×
232
233	def _migrate_project(self, project):	×
234	print("Migrating project %s/%s" % (self.gh_org, project))	×
235	# Create new branch
236	repo = self.github.repository(self.gh_org, project)	×
237	try:	×
238	repo.branch(self.gh_source_branch)	×
239	except NotFoundError:	×
240	print("Source branch non existing. Skipping...")	×
241	return	×
242	try:	×
243	repo.branch(self.gh_target_branch)	×
244	except NotFoundError:	×
245	pass	×
246	else:
247	print("Branch already exists. Skipping...")	×
248	return	×
249	root_contents = repo.directory_contents(	×
250	"",
251	self.gh_source_branch,
252	return_as=dict,
253	)
254	commit = self._create_metafiles(repo, root_contents)	×
255	repo.create_ref(	×
256	"refs/heads/%s" % self.gh_target_branch,
257	commit.sha,
258	)
259	# TODO: GitHub is returning 404
260	# self._make_default_branch(repo)
261
262	def do_migration(self, projects=None):	×
263	if not projects:	×
264	projects = oca_projects.get_repositories()	×
265	for project in projects:	×
266	self._migrate_project(project)	×
267
268
269	def get_parser():	×
270	parser = argparse.ArgumentParser(	×
271	description="Migrate one OCA branch from one version to another, "
272	"applying the needed transformations",
273	add_help=True,
274	)
275	parser.add_argument("source", help="Source branch (existing)")	×
276	parser.add_argument("target", help="Target branch (to create)")	×
277	parser.add_argument(	×
278	"-p",
279	"--projects",
280	dest="projects",
281	nargs="+",
282	default=[],
283	help="List of specific projects to migrate",
284	)
285	parser.add_argument(	×
286	"-e",
287	"--email",
288	dest="email",
289	help=(
290	"Provides an email address used to commit on GitHub if the one "
291	"associated to the GitHub account is not public"
292	),
293	)
294	parser.add_argument(	×
295	"-t",
296	"--target-org",
297	dest="target_org",
298	help=(
299	"By default, the GitHub organization used is OCA. This arg lets "
300	"you provide an alternative organization"
301	),
302	)
303	return parser	×
304
305
306	def main():	×
307	args = get_parser().parse_args()	×
308	migrator = BranchMigrator(	×
309	source=args.source,
310	target=args.target,
311	target_org=args.target_org,
312	email=args.email,
313	)
314	migrator.do_migration(projects=args.projects)	×
315
316
317	if __name__ == "__main__":	×
318	main()	×

OCA / maintainer-tools / 13228537945

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous