13228537945

Committed 09 Feb 2025 06:34PM UTC coverage: 35.131%. Remained the same

Build # 13228537945

Build Type

Pull #644

github

Committed by

web-flow

Commit Message

Merge 8b3aeb3fa into 16f1fc1f8

Pull Request Pull Request #644: Ignore archived projects

Run Details

437 of 1188 branches covered (36.78%)

645 of 1836 relevant lines covered (35.13%)

3.48 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

0.0

/tools/publish_modules.py

# Copyright 2019 Brainbean Apps (https://brainbeanapps.com)
# Copyright 2019 Tecnativa - Pedro M. Baeza
# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).
"""
This script helps you to add all the OCA repositories in batch in apps.odoo.com
platform. For now it's not adapted to other organization as it takes the
information from static mapping here. It shouldn't be too much difficult to
adapt this to other organizations.

It also serves as status scanner for knowing which repos are not yet being
automatically scanned (showing the conflict but ommited in case of simply
empty repo), and allowing to force the scan in that moment. Note that current
platform auto-scan all the repositories daily, so this operation is not really
needed except being in a hurry.

WARNING: This work is based on current platform implementation. This might
stop working if Odoo changes it through time.

Installation
============

It requires to have Python library `selenium`, that you can install regularly
through pip doing `sudo pip3 install selenium` or using specific OS packages.

It also requires chromedriver binary. If you are in Ubuntu or derivative, you
can do:

`sudo apt-get install chromium-chromedriver`

The 'chromedriver' executable must be in your PATH.

Configuration
=============

You can have a file called oca.cfg on the same folder of the script for
storing credentials parameters. You can generate an skeleton config running
this script for a first time.

The credentials are stored in the section [apps.odoo.com], with the names
"username" and "password", which are self-explanatory.

If not set, a prompt will ask you to enter user and password.

Usage
=====

oca-publish-modules [OPTIONS]

Options:
  --branch TEXT                   Limit to specific Odoo series. eg 11.0.
  --repository TEXT               Limit to a repository. eg contract.
  --registration / --no-registration
                                  Perform the registration of repositories.
  --status / --no-status          Retrieve the status of bad repositories.
  --force-scan / --no-force-scan  If auto-scan not activated, activate it and
                                  perform a scan in that moment.
  --scan-skip-empty / --scan-no-skip-empty
                                  Skip scan of empty repositories (no matter
                                  force scan value).
  --help                          Show the help.
"""

from __future__ import print_function

import logging
from getpass import getpass

import click
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait

from .config import read_config
from .oca_projects import get_repositories_and_branches, url

_logger = logging.getLogger(__name__)


@click.command()
@click.option(
    "--branch", "target_branch", help="Limit to specific Odoo series. eg 11.0."
)
@click.option(
    "--repository", "target_repository", help="Limit to a repository. eg contract."
)
@click.option("--org", help="GitHub organization (default=OCA)", default="OCA")
@click.option(
    "--registration/--no-registration",
    "do_registration",
    default=True,
    help="Perform the registration of repositories.",
)
@click.option(
    "--status/--no-status",
    "do_status",
    default=True,
    help="Retrieve the status of bad repositories.",
)
@click.option(
    "--force-scan/--no-force-scan",
    "force_scan",
    default=False,
    help="If auto-scan not activated, activate it and request "
    "apps.odoo.com to perform a scan in that moment.",
)
@click.option(
    "--scan-skip-empty/--scan-no-skip-empty",
    "scan_skip_empty",
    default=True,
    help="Skip scan of empty repositories (no matter force scan " "value).",
)
def main(
    target_branch,
    target_repository,
    org,
    do_registration,
    do_status,
    force_scan,
    scan_skip_empty,
):
    config = read_config()
    user = config.get("apps.odoo.com", "username")
    if not user:
        user = input("Odoo.com publisher account:")
    password = config.get("apps.odoo.com", "password")
    if not password:
        password = getpass(prompt="Odoo.com account password:")
    # Selenium options
    options = Options()
    options.headless = True
    driver = webdriver.Chrome(
        options=options,
    )
    login(driver, user, password)
    # First pass: register all repositories (if already registered, there
    # will be an immediate warning in the current browser page and won't
    # continue, so we can simply overwrite the value in the field).
    if do_registration:
        for repository, branch in get_repositories_and_branches():
            if target_branch and branch != target_branch:
                continue
            if target_repository and target_repository != repository:
                continue
            repository_url = url(repository) + "#" + branch
            print(
                "INFO: Adding %s#%s from %s... (if not yet present)"
                % (
                    repository,
                    branch,
                    repository_url,
                )
            )
            register_repository(driver, repository_url)
    # Second pass: check published state and try to publish if not yet done
    if do_status:
        for repository, branch in get_repositories_and_branches():
            if target_branch and branch != target_branch:
                continue
            if target_repository and target_repository != repository:
                continue
            # assume this query returns everything we need in one page
            driver.get(
                "https://apps.odoo.com/apps/dashboard/repos?"
                "search_in=url&search={org}/{repository}".format(
                    org=org, repository=repository
                )
            )
            scan_repository(
                driver,
                org,
                repository,
                branch,
                force_scan,
                scan_skip_empty,
            )


def login(driver, user, password):
    wait = WebDriverWait(driver, 10)
    driver.get(
        "https://www.odoo.com/web/login?redirect=%2Foauth2%2Fauth%2F%3Fscope"
        "%3Duserinfo%26redirect_uri%3Dhttps%253A%252F%252Fapps.odoo.com%252F"
        "auth_oauth%252Fsignin%26state%3D%257B%2522p%2522%253A%2B1%252C%2B"
        "%2522r%2522%253A%2B%2522%25252F%25252Fapps.odoo.com%25252Fapps%25"
        "22%252C%2B%2522d%2522%253A%2B%2522apps%2522%257D%26response_type%3D"
        "token%26client_id%3Da0a30d16-6095-11e2-9c70-002590a17fd8&scope=user"
        "info&mode=login&redirect_hostname=https%3A%2F%2Fapps.odoo.com&login="
    )
    login_field = driver.find_element_by_id("login")
    login_field.clear()
    login_field.send_keys(user)
    password_field = driver.find_element_by_id("password")
    password_field.clear()
    password_field.send_keys(password)
    login_button = driver.find_element_by_xpath(
        './/form[@action="/web/login"]//button[@type="submit"]'
    )
    login_button.click()
    wait.until(lambda driver: driver.current_url == "https://apps.odoo.com/apps")


def register_repository(driver, repository):
    driver.get("https://apps.odoo.com/apps/upload")
    url_field = driver.find_element_by_name("url")
    url_field.clear()
    url_field.send_keys(repository)
    submit_button = driver.find_element_by_id("apps_submit_repo_button")
    submit_button.click()


def scan_repository(driver, org, repository, branch, force_scan, scan_skip_empty):
    wait = WebDriverWait(driver, 300)
    for protocol in ("https", "ssh"):
        repository_url = url(repository, protocol=protocol, org_name=org) + "#" + branch
        try:
            item_container = driver.find_element_by_xpath(
                './/span[@id="repo_url" and text()="%s"]'
                "/ancestor::li[1]" % repository_url
            )
        except NoSuchElementException:
            pass
        else:
            break  # found
    else:
        # not found
        print(
            "WARNING: {org}/{repository}#{branch} "
            "not registered in this account.".format(
                org=org, repository=repository, branch=branch
            )
        )
        return
    try:
        error_item = item_container.find_element_by_xpath(
            './/div[@id="help_error"]/div/p',
        )
    except NoSuchElementException:
        error_item = False
    is_empty = False
    if error_item:
        is_empty = error_item.text.startswith("No module found in repository")
        if not is_empty:
            print("ERROR: %s:\n%s" % (repository_url, error_item.text))
    if force_scan and not (is_empty and scan_skip_empty):
        print("INFO: Doing the scan on %s" % repository_url)
        auto_scan_checkbox = item_container.find_element_by_xpath(
            './/input[@name="auto_scan"]'
        )
        if not auto_scan_checkbox.is_selected():
            auto_scan_checkbox.click()
            scan_link = item_container.find_element_by_class_name("js_repo_scan")
            scan_link.click()
            wait.until(
                lambda driver: driver.current_url
                == "https://apps.odoo.com/apps/dashboard/repos"
            )


if __name__ == "__main__":
    main()

1	# Copyright 2019 Brainbean Apps (https://brainbeanapps.com)
2	# Copyright 2019 Tecnativa - Pedro M. Baeza
3	# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).
4	"""
5	This script helps you to add all the OCA repositories in batch in apps.odoo.com
6	platform. For now it's not adapted to other organization as it takes the
7	information from static mapping here. It shouldn't be too much difficult to
8	adapt this to other organizations.
9
10	It also serves as status scanner for knowing which repos are not yet being
11	automatically scanned (showing the conflict but ommited in case of simply
12	empty repo), and allowing to force the scan in that moment. Note that current
13	platform auto-scan all the repositories daily, so this operation is not really
14	needed except being in a hurry.
15
16	WARNING: This work is based on current platform implementation. This might
17	stop working if Odoo changes it through time.
18
19	Installation
20	============
21
22	It requires to have Python library `selenium`, that you can install regularly
23	through pip doing `sudo pip3 install selenium` or using specific OS packages.
24
25	It also requires chromedriver binary. If you are in Ubuntu or derivative, you
26	can do:
27
28	`sudo apt-get install chromium-chromedriver`
29
30	The 'chromedriver' executable must be in your PATH.
31
32	Configuration
33	=============
34
35	You can have a file called oca.cfg on the same folder of the script for
36	storing credentials parameters. You can generate an skeleton config running
37	this script for a first time.
38
39	The credentials are stored in the section [apps.odoo.com], with the names
40	"username" and "password", which are self-explanatory.
41
42	If not set, a prompt will ask you to enter user and password.
43
44	Usage
45	=====
46
47	oca-publish-modules [OPTIONS]
48
49	Options:
50	--branch TEXT Limit to specific Odoo series. eg 11.0.
51	--repository TEXT Limit to a repository. eg contract.
52	--registration / --no-registration
53	Perform the registration of repositories.
54	--status / --no-status Retrieve the status of bad repositories.
55	--force-scan / --no-force-scan If auto-scan not activated, activate it and
56	perform a scan in that moment.
57	--scan-skip-empty / --scan-no-skip-empty
58	Skip scan of empty repositories (no matter
59	force scan value).
60	--help Show the help.
61	"""
62
63	from __future__ import print_function	×
64
65	import logging	×
66	from getpass import getpass	×
67
68	import click	×
69	from selenium import webdriver	×
70	from selenium.common.exceptions import NoSuchElementException	×
71	from selenium.webdriver.chrome.options import Options	×
72	from selenium.webdriver.support.ui import WebDriverWait	×
73
74	from .config import read_config	×
75	from .oca_projects import get_repositories_and_branches, url	×
76
77	_logger = logging.getLogger(__name__)	×
78
79
80	@click.command()	×
81	@click.option(	×
82	"--branch", "target_branch", help="Limit to specific Odoo series. eg 11.0."
83	)
84	@click.option(	×
85	"--repository", "target_repository", help="Limit to a repository. eg contract."
86	)
87	@click.option("--org", help="GitHub organization (default=OCA)", default="OCA")	×
88	@click.option(	×
89	"--registration/--no-registration",
90	"do_registration",
91	default=True,
92	help="Perform the registration of repositories.",
93	)
94	@click.option(	×
95	"--status/--no-status",
96	"do_status",
97	default=True,
98	help="Retrieve the status of bad repositories.",
99	)
100	@click.option(	×
101	"--force-scan/--no-force-scan",
102	"force_scan",
103	default=False,
104	help="If auto-scan not activated, activate it and request "
105	"apps.odoo.com to perform a scan in that moment.",
106	)
107	@click.option(	×
108	"--scan-skip-empty/--scan-no-skip-empty",
109	"scan_skip_empty",
110	default=True,
111	help="Skip scan of empty repositories (no matter force scan " "value).",
112	)
113	def main(
114	target_branch,
115	target_repository,
116	org,
117	do_registration,
118	do_status,
119	force_scan,
120	scan_skip_empty,
121	):
122	config = read_config()	×
123	user = config.get("apps.odoo.com", "username")	×
124	if not user:	×
125	user = input("Odoo.com publisher account:")	×
126	password = config.get("apps.odoo.com", "password")	×
127	if not password:	×
128	password = getpass(prompt="Odoo.com account password:")	×
129	# Selenium options
130	options = Options()	×
131	options.headless = True	×
132	driver = webdriver.Chrome(	×
133	options=options,
134	)
135	login(driver, user, password)	×
136	# First pass: register all repositories (if already registered, there
137	# will be an immediate warning in the current browser page and won't
138	# continue, so we can simply overwrite the value in the field).
139	if do_registration:	×
140	for repository, branch in get_repositories_and_branches():	×
141	if target_branch and branch != target_branch:	×
142	continue	×
143	if target_repository and target_repository != repository:	×
144	continue	×
145	repository_url = url(repository) + "#" + branch	×
146	print(	×
147	"INFO: Adding %s#%s from %s... (if not yet present)"
148	% (
149	repository,
150	branch,
151	repository_url,
152	)
153	)
154	register_repository(driver, repository_url)	×
155	# Second pass: check published state and try to publish if not yet done
156	if do_status:	×
157	for repository, branch in get_repositories_and_branches():	×
158	if target_branch and branch != target_branch:	×
159	continue	×
160	if target_repository and target_repository != repository:	×
161	continue	×
162	# assume this query returns everything we need in one page
163	driver.get(	×
164	"https://apps.odoo.com/apps/dashboard/repos?"
165	"search_in=url&search={org}/{repository}".format(
166	org=org, repository=repository
167	)
168	)
169	scan_repository(	×
170	driver,
171	org,
172	repository,
173	branch,
174	force_scan,
175	scan_skip_empty,
176	)
177
178
179	def login(driver, user, password):	×
180	wait = WebDriverWait(driver, 10)	×
181	driver.get(	×
182	"https://www.odoo.com/web/login?redirect=%2Foauth2%2Fauth%2F%3Fscope"
183	"%3Duserinfo%26redirect_uri%3Dhttps%253A%252F%252Fapps.odoo.com%252F"
184	"auth_oauth%252Fsignin%26state%3D%257B%2522p%2522%253A%2B1%252C%2B"
185	"%2522r%2522%253A%2B%2522%25252F%25252Fapps.odoo.com%25252Fapps%25"
186	"22%252C%2B%2522d%2522%253A%2B%2522apps%2522%257D%26response_type%3D"
187	"token%26client_id%3Da0a30d16-6095-11e2-9c70-002590a17fd8&scope=user"
188	"info&mode=login&redirect_hostname=https%3A%2F%2Fapps.odoo.com&login="
189	)
190	login_field = driver.find_element_by_id("login")	×
191	login_field.clear()	×
192	login_field.send_keys(user)	×
193	password_field = driver.find_element_by_id("password")	×
194	password_field.clear()	×
195	password_field.send_keys(password)	×
196	login_button = driver.find_element_by_xpath(	×
197	'.//form[@action="/web/login"]//button[@type="submit"]'
198	)
199	login_button.click()	×
200	wait.until(lambda driver: driver.current_url == "https://apps.odoo.com/apps")	×
201
202
203	def register_repository(driver, repository):	×
204	driver.get("https://apps.odoo.com/apps/upload")	×
205	url_field = driver.find_element_by_name("url")	×
206	url_field.clear()	×
207	url_field.send_keys(repository)	×
208	submit_button = driver.find_element_by_id("apps_submit_repo_button")	×
209	submit_button.click()	×
210
211
212	def scan_repository(driver, org, repository, branch, force_scan, scan_skip_empty):	×
213	wait = WebDriverWait(driver, 300)	×
214	for protocol in ("https", "ssh"):	×
215	repository_url = url(repository, protocol=protocol, org_name=org) + "#" + branch	×
216	try:	×
217	item_container = driver.find_element_by_xpath(	×
218	'.//span[@id="repo_url" and text()="%s"]'
219	"/ancestor::li[1]" % repository_url
220	)
221	except NoSuchElementException:	×
222	pass	×
223	else:
224	break # found	×
225	else:
226	# not found
227	print(	×
228	"WARNING: {org}/{repository}#{branch} "
229	"not registered in this account.".format(
230	org=org, repository=repository, branch=branch
231	)
232	)
233	return	×
234	try:	×
235	error_item = item_container.find_element_by_xpath(	×
236	'.//div[@id="help_error"]/div/p',
237	)
238	except NoSuchElementException:	×
239	error_item = False	×
240	is_empty = False	×
241	if error_item:	×
242	is_empty = error_item.text.startswith("No module found in repository")	×
243	if not is_empty:	×
244	print("ERROR: %s:\n%s" % (repository_url, error_item.text))	×
245	if force_scan and not (is_empty and scan_skip_empty):	×
246	print("INFO: Doing the scan on %s" % repository_url)	×
247	auto_scan_checkbox = item_container.find_element_by_xpath(	×
248	'.//input[@name="auto_scan"]'
249	)
250	if not auto_scan_checkbox.is_selected():	×
251	auto_scan_checkbox.click()	×
252	scan_link = item_container.find_element_by_class_name("js_repo_scan")	×
253	scan_link.click()	×
254	wait.until(	×
255	lambda driver: driver.current_url
256	== "https://apps.odoo.com/apps/dashboard/repos"
257	)
258
259
260	if __name__ == "__main__":	×
261	main()	×

OCA / maintainer-tools / 13228537945

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous