Send patches - preferably formatted by git format-patch - to patches at archlinux32 dot org.
summaryrefslogtreecommitdiff
path: root/archinstall/lib/general.py
AgeCommit message (Collapse)Author
2022-01-07Btrfs II (#838)Werner Llácer
* Btrfs with encrypted partitions. We have changed installer.mount_ordered_layout into a series of loops * open the encrypted devices * manage btrfs subvolumes * mount whatever * create kyefiles for encrypted volumes We have simplified the btrfs subvolume manager We merged the locale branch as it is needed here * We allow only the creation of keyfiles if the partition does not contain the root mount point. Also, adapt examples/only_hd to the new __init__.py Also, assorted flake8 warnings * Cleanup code * Naming schema for encrypted volumes revert global locale association (provisional) * We introduce the option of defining mount options in the partition dictionary. It has forced us to define two new entries in this dictionary: * format_options (formerly options) for mkfs options and * mount_options for mount -o ones. The different meaning of compress between partition and subvolumes is treated * Function lib/disk/btrfs.py mount_subvolume marked as deprecated Code cleanup. * format_options now filesystem.options * format_options now filesystem.format_options mount_options nof filesystem.mount_options * flake8 uncovered a slip in the code
2022-01-06Define an standard locale for all the commands executed (#827)Werner Llácer
* Define an standard locale for all the commands executed inside the application * Allow LC_ALL to be overriden during call * flake8 corrections
2022-01-06Added type annotations to all functions (#845)Anton Hvornum
* Added type annotations for 1/5 of the files. There's bound to be some issues with type miss-match, will sort that out later. * Added type hints for 4/5 of the code * Added type hints for 4.7/5 of the code * Added type hints for 5/5 of the code base * Split the linters into individual files This should help with more clearly show which runner is breaking since they don't share a single common name any longer. Also moved mypy settings into pyproject.toml * Fixed some of the last flake8 issues * Missing parameter * Fixed invalid lookahead types * __future__ had to be at the top * Fixed last flake8 issues
2022-01-02Improved command line argument parsing (#725)Werner Llácer
* An update to PR 715, making the handling of the *--mount-point* parameter less error prone. I added a synomym (accepting the name both with underscore and dash) and ignoring when no value specified I added it explicitly to the list to accept both the --parm value and --parm=value syntax DOES NOT check the contents of the parameter * Explicitly set all the know parameters * Define explictly all parameters. Make all non explicitly defined parameters behave as standard parameters, with on exception, names are not changed Some cleanup of the code In guided.py the reference to the dry_run parameter is updated to the standard naming convention for parameters * Linter with flake8. corrections * Linter with flake8. corrections (II) * Linter with flake8. corrections (and III) * Added --disk_layout argument. Was missing I moved its loading from guided.py to __init__.py as it happens to the other json related arguments * Better handling of errors during processing of the --disk_layouts parameter. I define a routine to read an store a JSON file or stream. Tested on disk_layout * Expand the former commit to all JSON file arguments * Moved the function we created to read json files/streams to general.py. Add some comments * flake8. A reference now unneded * The merge process for the dry-run argument was causing the issue, not solving it The del is just a cleanup for version upgrade without consequence (I hope) * flake8 warning * Correcting the last correction . Worked for old config files, but only for them * New parameter parsing algorithm. More flexible and accepts multiple arguments (optionallY) plus some documentation effort * flake8 warning. For once is significant ( != None to not None)
2021-12-31Bringing in changes from v2.3.1-dev branch. This fixes a selection of things ↵Anton Hvornum
mentioned in #819.
2021-12-02Fixing some mypy complaints (#780)Anton Hvornum
* Fixed some mypy issues regarding SysCommand* and logging * Fixed imports and undefined variable
2021-11-24Support encrypting multiple partitions (#759)Anton Hvornum
* Added support for storing disk encryption keyfiles and add them to a keyslot. * Added a luks2().add_key() function in order to inject a keyfile into a keyslot on a encrypted volume. * Simplified 'missing encryption password' logic in Filesystem(). Added a call to luks2().add_key() after the root-password is set on the volume, to add the keyfile in slot 2 * Adding in password handling in luks2().add_key(). It's required to enter a previous passphrase to unlock the volume and add a new keyslot. Also simplified the handling of partition in Installer().mount_ordered_layout() * Adding in encryption on all partitions except /boot when encryption is opted in * Removed setting size on Partition() as it's a read only value. No idea how Partition().size = size hasn't caused an issue before. Removed size=X argument to Partition() * Added a uniqueness to the loopdevice name. This should ensure that multiple encrypted volumes can be opened at the same time, except for Partition().detect_inner_filesystem() operations which can only happen one at a time since they share namespace. This should never be an issue since archinstall is single threaded and no concurrent operations can/should happen. * Added partprobe() as part of disk/helpers.py, added a /dev/ -> UUID mapper function called convert_device_to_uuid(path). Added a luks2().crypttab() function that sets up a /etc/crypttab entry. * Moved the responsability for telling archinstall to generate a keyfile from Filesystem() to user_interaction.py. This should in the future be a user-input based value, and not something the Filesystem() automatically dictates. * Added a retry mechanism to luks2().encrypt() to avoid having to re-start the installation when a device simply wasn't up yet. * Swapping UUID= lookup from loopdev to physdev.
2021-11-19source code improvementSteven Lee
2021-11-18Adding a cmd_history.txt log under /var/log/archinstall/ (#737)Anton Hvornum
* Adding a cmd_history.txt log under /var/log/archinstall/ to get a clear picture of which commands was executed.
2021-11-17Fixing broken encryption support in GRUB (#724)Anton Hvornum
* Added multiple `partprobe` calls and added a `.partprobe()` function on partitions, filesystem and blockdevice. * Adding retry attempts to all UUID related operations tied to the boot process * Tweaked logging for mounting and disk related operations * Removed potential SysCall exception disruptor causing exceptions to go by unnoticed * Increased the start position from 1MiB to 5MiB of /boot partition * Optimized the GRUB installation & config code * Improved Partition().uuid to never return None. Instead it will raise an exception if it can't get a PARTUUID within X retries with Y delay per attempt. * Increased sleep timer for partition uuid retrieval, because even with a 3 second sleep it wasn't long enough even on fast devices. * Make GRUB install to /dev/sda instead of /dev/sda1. * Added 10 retries for retreiving PARTUUID with a one second sleep. Instead of increasing the sleep simply add more retries until we find a good balance on slower disks.
2021-11-09Forgot to change the cls=JSON inside the recursive call of UNSAFE_JSONAnton Hvornum
2021-11-09Adding in storage of user supplied credentials. This separates credentials ↵Anton Hvornum
from user_configuration.json into user_credentials.json. As well as the JSON serializer will omit the credentials from the user_configuration.json by detecting ! in the dictionary keys (which is why they are important). UNSAFE_JSON will leave those keys in there, good for storing credentials in a separate file."
2021-11-05Cleaned up all flake8 issues/warnings. Did some code cleaning as well, ↵Anton Hvornum
mostly how we called things in guided.py but also some SysCommand calls
2021-11-01Fixed general.py flake8 issues.Anton Hvornum
2021-09-06Merged in master.Anton Hvornum
2021-07-16Replace for loops with comprehensionsmonosans
2021-06-10Added wipe support to layout definitions. Also changed default start ↵Anton Hvornum
positions of partitions to 1MiB in.
2021-06-10Merging in latest changes from master.Anton Hvornum
2021-06-10Merge pull request #537 from archlinux/torxed-fix-517Anton Hvornum
Introduces the use of HTTP mirrors additionally
2021-06-10Merge pull request #538 from archlinux/torxed-fix-536Anton Hvornum
Attempt to fix issue regarding broken output
2021-06-06Added a json.dumps() helper that wraps JSON cls. Also tweaked the logic for ↵Anton Hvornum
the size creation so that they don't overlap
2021-06-02Wrong variable nameAnton Hvornum
2021-06-02Adding more granular debuggingAnton Hvornum
2021-06-02Attempt to fix issue regarding broken outputAnton Hvornum
Since pacman and some other commands these days write multiple lines and goes back and fourth, it's not reliable to say the "peaked" line is a single line. And if it's not, it will make the output look like garbage. So instead, we'll write any output - as is - and let the command deal with fancy printing.
2021-06-01Windows fix + Sorting based on listAnton Hvornum
This fix introduces changes so that development can be done (and tested) on other platforms than Linux. This is a convenience fix and shouldn't break anything (simply a few Linux-specific imports that have moved into the functions where they are used). This commit also introduces sorting based on a list of priorities (where the default will be last if not matched).
2021-05-22Since SysCommand() wraps SysCommandWorker(), but SysCommand() doesn't use ↵Anton Hvornum
contexts, we have to flush the output manually here to avoid newlines not being before the next output.
2021-05-20Use {installation.target} in custom commands handlerDylan Taylor
2021-05-20Extract custom-commands function into general.pyDylan Taylor
2021-05-19very minor whitespace fixesDylan Taylor
2021-05-19Added support for getting configuration from a config file (#364)Yash Tripathi
* added support for ingesting config * fixed condition to check key in dictionary * Removed redundant code, profile and desktop keys are now optional * Added base-config.json and support for pulling credentials from .env * added base config file and env file for users credentials * added silent install switch * added python-dotenv as a dependency * Updated Readme to include argparse changes as well as config ingestion * Updated Readme to include argparse changes as well as config ingestion * fixed typo in pyproject.toml * Replaced the magic __builtin__ global variable. This should fix mypy complaints while still retaining the same functionality, kinda. It's less automatic but it's also less of dark magic, which makes sense for anyone but me. * Fixes string index error. * Quotation error. * fixed initializing --script argument * added python-dotenv as a dependency * Installation can't be silent if config is not passed * fixed silent install help * fixed condition for ask_user_questions * reverted to creating profile object properly * Cleaned up and incorporated suggestions * added Profile import * added condition if Profile is null * fixed condition * updated parsing vars from argparse * removed loading users from .env * Reworking SysCommand & Moving to localectl for locale related activities (#4) * Moving to `localectl` rather than local file manipulation *(both for listing locales and setting them)*. * Swapped `loadkeys` for localectl. * Renamed `main` to `maim` in awesome profile. * Created `archinstall.Boot(<installation>)` which spawns a `systemd-nspawn` container against the installation target. * Exposing systemd.py's internals to archinstall global scope. * Re-worked `SysCommand` completely, it's now a wrapper for `SysCommandWorker` which supports interacting with the process in a different way. `SysCommand` should behave just like the old one, for backwards compatibility reasons. This fixes #68 and #69. * `SysCommand()` now has a `.decode()` function that defaults to `UTF-8`. * Adding back peak_output=True to pacstrap. Co-authored-by: Anton Hvornum <anton.feeds@gmail.com> Co-authored-by: Dylan Taylor <dylan@dylanmtaylor.com> Co-authored-by: Anton Hvornum <anton@hvornum.se> Co-authored-by: Anton Hvornum <anton.feeds@gmail.com> * fixed indent * removed redundant import * removed duplicate import * removed duplicate import Co-authored-by: Anton Hvornum <anton.feeds@gmail.com> Co-authored-by: Anton Hvornum <anton@hvornum.se> Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com>
2021-05-19Reworking SysCommand & Moving to localectl for locale related activitiesAnton Hvornum
* Moving to `localectl` rather than local file manipulation *(both for listing locales and setting them)*. * Swapped `loadkeys` for localectl. * Renamed `main` to `maim` in awesome profile. * Created `archinstall.Boot(<installation>)` which spawns a `systemd-nspawn` container against the installation target. * Exposing systemd.py's internals to archinstall global scope. * Re-worked `SysCommand` completely, it's now a wrapper for `SysCommandWorker` which supports interacting with the process in a different way. `SysCommand` should behave just like the old one, for backwards compatibility reasons. This fixes #68 and #69. * `SysCommand()` now has a `.decode()` function that defaults to `UTF-8`. * Adding back peak_output=True to pacstrap. Co-authored-by: Anton Hvornum <anton.feeds@gmail.com> Co-authored-by: Dylan Taylor <dylan@dylanmtaylor.com>
2021-05-15Perform refactoring to PEP 8 naming conventionsDylan Taylor
2021-05-15Fix mutable default argumentsDylan Taylor
https://docs.python-guide.org/writing/gotchas/#mutable-default-arguments
2021-05-15More formatting fixes to satisfy PEP 8Dylan Taylor
2021-05-15f-string fixesDylan Taylor
2021-05-15Whitespace changesDylan Taylor
2021-05-15Resolve a few more PEP issues in general.pyDylan Taylor
2021-05-15general.py formattingDylan Taylor
2021-05-15Many more manual changesDylan Taylor
2021-05-12some type hint fixes and a bad catch fixadvaithm
2021-04-27Moving away from custom log levels, to something that's well defined. (#360)Anton Hvornum
* Moving away from custom log levels, to something that's well defined. * Added backward compability to log() as well. * Added an option to force log messages out on screen even if the level is below the log level threashold. * Added force log messages when wrong notation is used. * Added some more length to the deprecated message * Swapped all log levels to use logging.<level> instead. Co-authored-by: Anton Hvornum <anton.feeds@gmail.com>
2021-04-17Merging in master to feature branch.Anton Hvornum
2021-04-13Modifying to enable piping in custom environment variables. This is ↵Anton Hvornum
required to disable systemd coloring and paging when querying for service states. Otherwise it returns unreliable data that can cause hanging.
2021-04-13Closing forked process PID in order to close any ramining open file handles.Anton Hvornum
2021-04-04Removed debugging, added an additional 'move back to beginning' and print ↵Anton Hvornum
the peak again. Added this to any pacstrap call.
2021-04-04Added some debuggingAnton Hvornum
2021-04-04Added stripping of peak output to avoid new lines forming.Anton Hvornum
2021-04-04Added error handling to sys_command's peak function.Anton Hvornum
2021-04-04Moved import due to circular imports.Anton Hvornum
2021-04-04Added functionality towards #155. It's not a progress bar per sae, but it ↵Anton Hvornum
will show the last line outputted by commands enabling peaking of the ongoing process.