Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added support for ranged compiler versions for install flag #85

Open
wants to merge 13 commits into
base: dev
Choose a base branch
from
Open
16 changes: 10 additions & 6 deletions .github/workflows/linux-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,16 +15,20 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python:
- "3.7"
- "3.8"
- "3.9"
- "3.10"
type: ["solc_upgrade", "linux","solc"]
steps:
- uses: actions/checkout@v1
- name: Set up Python 3.6
uses: actions/setup-python@v1
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: 3.6
python-version: ${{ matrix.python }}
- name: Install solc-select
run: |
sudo python3 setup.py install
sudo pip install .
solc-select install all
- name: Run Tests
env:
Expand Down
16 changes: 10 additions & 6 deletions .github/workflows/mac-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,19 +15,23 @@ jobs:
runs-on: macos-latest
strategy:
matrix:
python:
- "3.7"
- "3.8"
- "3.9"
- "3.10"
type: ["solc_upgrade", "macos", "solc"]
steps:
- uses: actions/checkout@v1
- name: Set up Python 3.7.12
uses: actions/setup-python@v1
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: 3.7.12
python-version: ${{ matrix.python }}
- name: Install solc-select
run: |
sudo python3 setup.py install
sudo pip install .
solc-select install all
- name: Run Tests
env:
TEST_TYPE: ${{ matrix.type }}
run: |
bash scripts/test_${TEST_TYPE}.sh
bash scripts/test_${TEST_TYPE}.sh
37 changes: 37 additions & 0 deletions .github/workflows/pip-audit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: Scan dependencies for vulnerabilities with pip-audit

on:
push:
branches: [ "dev" ]
pull_request:
branches: [ "dev" ]
schedule:
- cron: "0 12 * * *"

jobs:
pip-audit:
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v3

- name: Install Python
uses: actions/setup-python@v4
with:
python-version: "3.x"

- name: Install project
run: |
python -m venv /tmp/pip-audit-env
source /tmp/pip-audit-env/bin/activate

python -m pip install --upgrade pip
python -m pip install .


- name: Run pip-audit
uses: trailofbits/gh-action-pip-audit@v0.0.4
with:
virtual-environment: /tmp/pip-audit-env

18 changes: 11 additions & 7 deletions .github/workflows/windows-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,19 +12,23 @@ on:

jobs:
tests:
runs-on: windows-latest
runs-on: windows-2022
strategy:
matrix:
python:
- "3.7"
- "3.8"
- "3.9"
- "3.10"
type: ["windows","solc"]
steps:
- uses: actions/checkout@v1
- name: Set up Python 3.6
uses: actions/setup-python@v1
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: 3.6
python-version: ${{ matrix.python }}
- name: Install solc-select
run: |
python3 setup.py install --user
pip install . --user
solc-select install all
- name: Run Tests
env:
Expand Down
28 changes: 28 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,19 @@ Feel free to stop by our [Slack channel](https://empirehacking.slack.com/) for h

Uninstall other installations of solc on your machine. `solc-select` re-installs solc binaries for your operating system and acts as a wrapper for solc. With duplicate solc installations, this may result in your `solc` version not being up to date.

### "Unsupported Platform" on Windows

The solc-select version that supports Windows is currently in beta. Uninstall `solc-select` through `pip3 uninstall solc-select` and run

```bash
pip install solc-select==1.0.0b1
```

Alternatively, for the most up-to-date version, clone this repository and run
```bash
pip install . --user
```

## Known Issues

### `SSL: CERTIFICATE_VERIFY_FAILED` on running `solc-select` commands [investigation ongoing]
Expand All @@ -99,6 +112,21 @@ Try downgrading to `solc-select version 0.2.0`.

Our `0.2.1` version of `solc-select` pulls older Linux binaries from [crytic/solc](https://github.com/crytic/solc) which seems to have introduced unexpected behavior in certain instances.

### `solc-select` version changes, but `solc --version does not match`

Users seem to be experiencing situations in which the following command is successful:
```
solc-select use <version>
```
However, when running the following command, it points to an older version of Solidity.
```
solc --version
```

`solc-select` is intended to work with custom binaries. This means that Solidity installed through other means (i.e: `brew install solidity`) will _not_ work!.

Uninstall other versions Solidity from your computer.

## License

`solc-select` is licensed and distributed under the [AGPLv3](LICENSE) license. [Contact us](mailto:opensource@trailofbits.com) if you’re looking for an exception to the terms.
3 changes: 3 additions & 0 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,7 @@
"solc = solc_select.__main__:solc",
]
},
install_requires=[
'pysha3'
]
)
2 changes: 1 addition & 1 deletion solc_select/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ def solc_select() -> None:
)
parser_install.add_argument(
INSTALL_VERSIONS,
help='specific versions you want to install "0.4.25" or "all"',
help='specific versions you want to install "0.4.25" or "0.4.24-0.4.25" for multiple versions or "all"',
nargs="*",
default=list(),
type=valid_install_arg,
Expand Down
11 changes: 10 additions & 1 deletion solc_select/constants.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
import os
from pathlib import Path

# DIRs path
HOME_DIR = Path.home()
if "VIRTUAL_ENV" in os.environ:
HOME_DIR = Path(os.environ["VIRTUAL_ENV"])
else:
HOME_DIR = Path.home()
SOLC_SELECT_DIR = HOME_DIR.joinpath(".solc-select")
ARTIFACTS_DIR = SOLC_SELECT_DIR.joinpath("artifacts")

Expand All @@ -16,3 +20,8 @@
WINDOWS_AMD64 = "windows-amd64"

EARLIEST_RELEASE = {"macosx-amd64": "0.3.6", "linux-amd64": "0.4.0", "windows-amd64": "0.4.5"}

# Regexes
SOLC_VERSION_REGEX = r"[\d]+.[\d]+.[\d]+"
SOLC_VERSION_RANGE_REGEX = f"({SOLC_VERSION_REGEX}){{1}}-({SOLC_VERSION_REGEX}){{1}}"
INSTALL_VERSIONS_INPUT_REGEX = f"^({SOLC_VERSION_RANGE_REGEX})|({SOLC_VERSION_REGEX})$"
90 changes: 68 additions & 22 deletions solc_select/solc_select.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import argparse
import hashlib
import sha3
import json
from zipfile import ZipFile
import os
Expand Down Expand Up @@ -60,12 +61,16 @@ def installed_versions() -> [str]:
]


def install_artifacts(versions: [str]) -> None:
def install_artifacts(versions: [str]) -> bool:
releases = get_available_versions()
match, version_from, version_to = should_install_artifacts_range(versions)

for version, artifact in releases.items():
if "all" not in versions:
if versions and version not in versions:
if match:
if not version_from <= StrictVersion(version) <= version_to:
continue
elif versions and version not in versions:
continue

(url, _) = get_url(version, artifact)
Expand All @@ -87,6 +92,7 @@ def install_artifacts(versions: [str]) -> None:
else:
Path.chmod(artifact_file_dir.joinpath(f"solc-{version}"), 0o775)
print(f"Version '{version}' installed.")
return True


def is_older_linux(version: str) -> bool:
Expand All @@ -107,7 +113,7 @@ def verify_checksum(version: str) -> None:
# calculate sha256 and keccak256 checksum of the local file
with open(ARTIFACTS_DIR.joinpath(f"solc-{version}", f"solc-{version}"), "rb") as f:
sha256_factory = hashlib.sha256()
keccak_factory = hashlib.sha3_256()
keccak_factory = sha3.keccak_256()

# 1024000(~1MB chunk)
for chunk in iter(lambda: f.read(1024000), b""):
Expand All @@ -116,14 +122,14 @@ def verify_checksum(version: str) -> None:

local_sha256_file_hash = f"0x{sha256_factory.hexdigest()}"
local_keccak256_file_hash = f"0x{keccak_factory.hexdigest()}"

if sha256_hash != local_sha256_file_hash and keccak256_hash != local_keccak256_file_hash:
if sha256_hash != local_sha256_file_hash or keccak256_hash != local_keccak256_file_hash:
raise argparse.ArgumentTypeError(
f"Error: Checksum mismatch {soliditylang_platform()} - {version}"
)


def get_soliditylang_checksums(version: str):
def get_soliditylang_checksums(version: str) -> (str, str):
(_, list_url) = get_url(version=version)
list_json = urllib.request.urlopen(list_url).read()
builds = json.loads(list_json)["builds"]
Expand Down Expand Up @@ -157,40 +163,55 @@ def switch_global_version(version: str, always_install: bool) -> None:
print("Switched global version to", version)
elif version in get_available_versions():
if always_install:
install_artifacts(version)
install_artifacts([version])
switch_global_version(version, always_install)
else:
raise argparse.ArgumentTypeError(f"'{version}' must be installed prior to use.")
else:
raise argparse.ArgumentTypeError(f"Unknown version '{version}'")


def valid_version(version: str) -> str:
match = re.search(r"^(\d+)\.(\d+)\.(\d+)$", version)

if match is None:
raise argparse.ArgumentTypeError(f"Invalid version '{version}'.")
def valid_version(install_input: str, string_version: bool = True) -> str:
match = re.search(INSTALL_VERSIONS_INPUT_REGEX, install_input)

if StrictVersion(version) < StrictVersion(EARLIEST_RELEASE[soliditylang_platform()]):
raise argparse.ArgumentTypeError(
f"Invalid version - only solc versions above '{EARLIEST_RELEASE[soliditylang_platform()]}' are available"
)
if match is None or (not match.group(4) and string_version):
raise argparse.ArgumentTypeError(f"Invalid version '{install_input}'.")

(_, list_url) = get_url()
list_json = urllib.request.urlopen(list_url).read()
latest_release = json.loads(list_json)["latestRelease"]
if StrictVersion(version) > StrictVersion(latest_release):
raise argparse.ArgumentTypeError(
f"Invalid version '{latest_release}' is the latest available version"
)

return version
def check_available_version(version: str):
if StrictVersion(version) < StrictVersion(EARLIEST_RELEASE[soliditylang_platform()]):
raise argparse.ArgumentTypeError(
f"Invalid version - only solc versions above '{EARLIEST_RELEASE[soliditylang_platform()]}' are available"
)

if StrictVersion(version) > StrictVersion(latest_release):
raise argparse.ArgumentTypeError(
f"Invalid version '{latest_release}' is the latest available version"
)

if match.group(4):
check_available_version(install_input)
else:
version_from = match.group(2)
version_to = match.group(3)
check_available_version(version_from)
check_available_version(version_to)

if StrictVersion(version_from) == StrictVersion(version_to):
return version_from
elif StrictVersion(version_from) > StrictVersion(version_to):
return f"{version_to}-{version_from}"

return install_input


def valid_install_arg(arg: str) -> str:
if arg == "all":
return arg
return valid_version(arg)
return valid_version(arg, False)


def get_installable_versions() -> [str]:
Expand Down Expand Up @@ -222,3 +243,28 @@ def soliditylang_platform() -> str:
else:
raise argparse.ArgumentTypeError("Unsupported platform")
return platform


def should_install_artifacts_range(versions: [str]) -> (bool, StrictVersion, StrictVersion):
match: bool = False
version_from: StrictVersion = StrictVersion("")
version_to: StrictVersion = StrictVersion("")

for version in versions:
curr_match = re.search(SOLC_VERSION_RANGE_REGEX, version)
if curr_match:
new_version_from = StrictVersion(curr_match.group(1))
new_version_to = StrictVersion(curr_match.group(2))

if match:
if new_version_from < version_from:
version_from = new_version_from

if new_version_to > version_to:
version_to = new_version_to
else:
version_from = new_version_from
version_to = new_version_to
match = True

return match, version_from, version_to