Send patches - preferably formatted by git format-patch - to patches at archlinux32 dot org.
summaryrefslogtreecommitdiff
path: root/archinstall/lib/disk/filesystem.py
AgeCommit message (Collapse)Author
2023-04-19PyParted and a large rewrite of the underlying partitioning (#1604)Daniel Girtler
* Invert mypy files * Add optional pre-commit hooks * New profile structure * Serialize profiles * Use profile instead of classmethod * Custom profile setup * Separator between back * Support profile import via url * Move profiles module * Refactor files * Remove symlink * Add user to docker group * Update schema description * Handle list services * mypy fixes * mypy fixes * Rename profilesv2 to profiles * flake8 * mypy again * Support selecting DM * Fix mypy * Cleanup * Update greeter setting * Update schema * Revert toml changes * Poc external dependencies * Dependency support * New encryption menu * flake8 * Mypy and flake8 * Unify lsblk command * Update bootloader configuration * Git hooks * Fix import * Pyparted * Remove custom font setting * flake8 * Remove default preview * Manual partitioning menu * Update structure * Disk configuration * Update filesystem * luks2 encryption * Everything works until installation * Btrfsutil * Btrfs handling * Update btrfs * Save encryption config * Fix pipewire issue * Update mypy version * Update all pre-commit * Update package versions * Revert audio/pipewire * Merge master PRs * Add master changes * Merge master changes * Small renaming * Pull master changes * Reset disk enc after disk config change * Generate locals * Update naming * Fix imports * Fix broken sync * Fix pre selection on table menu * Profile menu * Update profile * Fix post_install * Added python-pyparted to PKGBUILD, this requires [testing] to be enabled in order to run makepkg. Package still works via python -m build etc. * Swaped around some setuptools logic in pyproject Since we define `package-data` and `packages` there should be no need for: ``` [tool.setuptools.packages.find] where = ["archinstall", "archinstall.*"] ``` * Removed pyproject collisions. Duplicate definitions. * Made sure pyproject.toml includes languages * Add example and update README * Fix pyproject issues * Generate locale * Refactor imports * Simplify imports * Add profile description and package examples * Align code * Fix mypy * Simplify imports * Fix saving config * Fix wrong luks merge * Refactor installation * Fix cdrom device loading * Fix wrongly merged code * Fix imports and greeter * Don't terminate on partprobe error * Use specific path on partprobe from luks * Update archinstall/lib/disk/device_model.py Co-authored-by: codefiles <11915375+codefiles@users.noreply.github.com> * Update archinstall/lib/disk/device_model.py Co-authored-by: codefiles <11915375+codefiles@users.noreply.github.com> * Update github workflow to test archinstall installation * Update sway merge * Generate locales * Update workflow --------- Co-authored-by: Daniel Girtler <girtler.daniel@gmail.com> Co-authored-by: Anton Hvornum <anton@hvornum.se> Co-authored-by: Anton Hvornum <anton.feeds+github@gmail.com> Co-authored-by: codefiles <11915375+codefiles@users.noreply.github.com>
2023-03-29Save encryption configuration (#1672)Daniel Girtler
* Save encryption configuration * Fix deserialization problem * Added .part_uuid to MapperDev --------- Co-authored-by: Daniel Girtler <girtler.daniel@gmail.com> Co-authored-by: Anton Hvornum <anton@hvornum.se> Co-authored-by: Anton Hvornum <anton.feeds+github@gmail.com>
2023-02-27Patch for 1557 (#1645)Anton Hvornum
* Attempting a retry-attempt on the broken part of lsblk * Improved logging * Adding a retry to Partition._call_lsblk() * Added error checks if lsblk returns nothing, also handles empty Partition().info instance. * Added missing check of disk encryption is None or not. * Added tweak to catching output from lsblk.stderr * Added missing check of disk encryption is None or not. * Fixed a logic test for empty lsblk info * Fixed instances of None being interated * Added some errro handling for weird block devices * Fixed flake8 * Added /etc/vconsole.conf generation in Installer.mkinitcpio() as it's a dependency for it to generate properly without errors. Otherwise we'll get ==> ERRROR: file not found: '/etc/vconsole.conf' * Prep for tagging RC1 of 2.5.3 * Corrected helpers.py get_blockdevice_info() to deal with empty lsblk results
2022-11-11New encryption menu (#1520)Daniel Girtler
* New encryption menu Co-authored-by: Daniel Girtler <girtler.daniel@gmail.com> Co-authored-by: Anton Hvornum <anton@hvornum.se>
2022-10-12Removed debug outputAnton Hvornum
2022-10-12Updating Partition() information after mount and unmount. (#1508)Anton Hvornum
* Updating Partition() information after mount and unmount. * Cleaned up raw_parted() to gracefully output relevant partition error information. * Adding timestmap to cmd_history.txt as it's impossible to debug properly otherwise * Adding more verbose debugging information * Reinstating the lsblk retry code for PARTUUID and UUID on Partition()'s information * Added error handling for JSON parsing
2022-08-30Fixing issue where blkid causes SysCallException (#1445)Anton Hvornum
* Moving a partprobe() call to better allow for cache updates * Trying to improve Partition()._fetch_information() * Removed a sleep() for debugging purposes * Tweaked a sleep
2022-07-26Cleanup partition (#1333)Daniel Girtler
* Cleanup partition * Update * Remove unused method * Update partitioning * Update * Update * Fix mypy Co-authored-by: Daniel Girtler <girtler.daniel@gmail.com>
2022-06-07Update blockdevice (#1289)Daniel Girtler
* Update blockdevice class Co-authored-by: Daniel Girtler <girtler.daniel@gmail.com>
2022-05-29Fix typos (#1265)Kian-Meng Ang
2022-05-27Fix mixup of mount options and fstype during mount (#1258)Anton Hvornum
Fixed issue where `options` wasn't passed as `options=options` and got mixed up as fs-type.
2022-05-27Removed debug informationAnton Hvornum
2022-05-27Creates partition label on drives with zero partitions before adding ↵Anton Hvornum
partitions (#1256) * If BlockDevice.partitions is zero, we set a new partition label to ensure new drives are setup correctly * Added a sleep after mklabel. * Removed redundant log message. * Tweaked log message from print() to log(). * Trying PARTUUID in reusage of partitions. * Debugging.
2022-05-27Fix dual-booting (#1250)Anton Hvornum
# Fixes * Optimized partition lookups * Fixed re-use of partition UUID's * `BlockDevice().get_partition()` now supports looking up both `PARTUUID` and `UUID` for a partition under itself * Partitions listed in `--disk-layout` that doesn't have a PARTUUID/UUID should no longer cause an exception, but instead logs a warning and they will simply be ignored * `Filesystem().add_partition()` now handles `DiskError` raised by `partition.part_uuid` * Fixed issue on normal partitions where the device was not properly frozen in `lambda` calls, meaning two or more mount-points shared the same `device_instance`. * Lowered global `DISK_RETRY_ATTEMPTS` to 5, as the timeouts are linear *(`range(DISK_RETRY_ATTEMPTS) * DISK_TIMEOUTS`)*
2022-05-26Rework btrfs handling (#1234)Anton Hvornum
* Restructuring btrfs.py into lib/btrfs/*.py * Reworking how BTRFS subvolumes get represented, and worked with. Subvolumes are now their own entity which can be used to access it's information, parents or mount location. * Added BtrfsSubvolume.partition and other stuff. * Reworking the way luks2().unlock and .format() returns device instances. They should now return BTRFSSubvolume where appropriate. * Fixed a missing import * Fixed an issue where mkfs.btrfs wouldn't trigger due to busy disk. * Fixing subvol mounting without creating a fake instance. * Added creation of mountpint for btrfs subvolume * Fixed root detection * Re-worked mounting into a queue system using frozen mounting calls using lambda * Removed old mount_subvolume() function * Removed get_subvolumes_from_findmnt() * Fixed Partition().subvolumes iteration * Adding .root to BtrfsSubvolume * Fixed issue in SysCommandWorker where log output would break and crash execution due to cmd being a string vs list * Changed return-value from MapperDev.mountpoint to pathlib.Path
2022-05-18Added a HSM menu entry (#1196)Anton Hvornum
* Added a HSM menu entry, but also a safety check to make sure a FIDO device is connected * flake8 complaints * Adding FIDO lookup using cryptenroll listing * Added systemd-cryptenroll --fido2-device=list * Removed old _select_hsm call * Fixed flake8 complaints * Added support for locking and unlocking with a HSM * Removed hardcoded paths in favor of PR merge * Removed hardcoded paths in favor of PR merge * Fixed mypy complaint * Flake8 issue * Added sd-encrypt for HSM and revert back to encrypt when HSM is not used (stability reason) * Added /etc/vconsole.conf and tweaked fido2_enroll() to use the proper paths * Spelling error * Using UUID instead of PARTUUID when using HSM. I can't figure out how to get sd-encrypt to use PARTUUID instead. Added a Partition().part_uuid function. Actually renamed .uuid to .part_uuid and created a .uuid instead. * Adding missing package libfido2 and removed tpm2-device=auto as it overrides everything and forces password prompt to be used over FIDO2, no matter the order of the options. * Added some notes to clarify some choices. * Had to move libfido2 package install to later in the chain, as there's not even a base during mounting :P
2022-05-16Don't assume children exists (#1193)Anton Hvornum
2022-04-26Optimized a bunch of partprobe calls. (#1088)Anton Hvornum
* Optimized a bunch of partprobe calls. Namely fixed sleep calls, added optional path to the general archinstall.partprobe() call. And fixed some error handling in a few places which should tell us where #1083 might be going wrong. * Fixed some flake8 complaints * Fixed sleep having a min() of 0.1 or given value. * Fixed sleep having a correct range variable. * Fixed sleep logic to use max() instead of min() as it will never use the higer sleep values otheride
2022-02-28Fix (some) mypy things (#996)Daniel
* Fix mypy things * Fix flake8 Co-authored-by: Daniel Girtler <girtler.daniel@gmail.com>
2022-02-06Update nationalization (#944)Daniel
* Update nationalization * Update translations * Finish german translation * Fix errors #943 * Add remaining translations * Fix alignment in menu * Update README * Update translations: * Fix flake8 * Update tz function Co-authored-by: Daniel Girtler <girtler.daniel@gmail.com>
2022-02-04disk layout: allow to omit partition "start" option to start from previous ↵Victor Gavro
partition end (#895) * disk layout: allow to omit partition "start" option to start from previous partition end * mixed tabs/spaces fixes * Update filesystem.py Co-authored-by: Anton Hvornum <anton.feeds+github@gmail.com>
2022-02-01Issue923 (#927)Werner Llácer
* Correct definition of btrfs standard layout * Solving issue 923. Standarize keyword "wipe" to signal the need of formatting a FS * flake8 complains
2022-01-25Fixing the 'device_instance' being None in some partition places (#902)Anton Hvornum
* Added a new return value from add_partition. Also added an exception to make sure `add_partition` can't continue silently * Added a log of debugging to add_partition * Removed a blank line (flake8) * Misconfigured variable * Added some more debugging information to partprobe * FIX: disk layout: partprobe should be called and checked only for target device (#896) * disk layout: partprobe should be called and checked only for target device * disk layout: partprobe: removed unnecessary bash subprocess * Properly defined BlockDevice() on Partition() creation. Also made sure mount-checks got some rrro handling and non-block devices should no longer attempt to return a size Co-authored-by: Anton Hvornum <anton.feeds@gmail.com> Co-authored-by: Victor Gavro <vgavro@gmail.com>
2022-01-25Make text for getting filesystems more grammatically correct (#903)Dylan M. Taylor
* Make text for getting filesystems more grammatically correct Use 'Retry attempt {count} of 10.' instead of Waiting for the {count} time". It's more grammatically correct since we're not doing '1st', '2nd', etc. * Maybe mention waiting still
2022-01-25FIX: disk layout: partprobe should be called and checked only for target ↵Victor Gavro
device (#896) * disk layout: partprobe should be called and checked only for target device * disk layout: partprobe: removed unnecessary bash subprocess
2022-01-25Removed last lsblk to grab PARTUUID (#901)Anton Hvornum
Co-authored-by: Anton Hvornum <anton.feeds@gmail.com>
2022-01-07Btrfs II (#838)Werner Llácer
* Btrfs with encrypted partitions. We have changed installer.mount_ordered_layout into a series of loops * open the encrypted devices * manage btrfs subvolumes * mount whatever * create kyefiles for encrypted volumes We have simplified the btrfs subvolume manager We merged the locale branch as it is needed here * We allow only the creation of keyfiles if the partition does not contain the root mount point. Also, adapt examples/only_hd to the new __init__.py Also, assorted flake8 warnings * Cleanup code * Naming schema for encrypted volumes revert global locale association (provisional) * We introduce the option of defining mount options in the partition dictionary. It has forced us to define two new entries in this dictionary: * format_options (formerly options) for mkfs options and * mount_options for mount -o ones. The different meaning of compress between partition and subvolumes is treated * Function lib/disk/btrfs.py mount_subvolume marked as deprecated Code cleanup. * format_options now filesystem.options * format_options now filesystem.format_options mount_options nof filesystem.mount_options * flake8 uncovered a slip in the code
2022-01-06Added type annotations to all functions (#845)Anton Hvornum
* Added type annotations for 1/5 of the files. There's bound to be some issues with type miss-match, will sort that out later. * Added type hints for 4/5 of the code * Added type hints for 4.7/5 of the code * Added type hints for 5/5 of the code base * Split the linters into individual files This should help with more clearly show which runner is breaking since they don't share a single common name any longer. Also moved mypy settings into pyproject.toml * Fixed some of the last flake8 issues * Missing parameter * Fixed invalid lookahead types * __future__ had to be at the top * Fixed last flake8 issues
2021-12-31Btrfs2 (#787)Werner Llácer
* All the changes needed to make btrfs subvolumes work. It boils down to two points; the handling of the addressing of subvolumes re. physical partitions, and the small changes at the bootloader level * We added a new script only_hd for testing purposes. It only handles hadrd drive management * restoring an escape hatch during subvolume processing * hipercommented manage_btrfs_subvolumes * Ready to be able to select and process options in subvolume mounting * Separte nodatacow processing * Solving a flake8 complain * Use of bind names @ get_filesystem_type * compress mount option bypass * Preparations for encryption handling * Compatibility to master version re. encrypted btrfs volumes * Now we can create subvolumes and mountpoints inside an encrypted btrfs partition * changes for entries file generation with systemd-bootctl * flake8 corrections plus some comments Co-authored-by: Anton Hvornum <anton@hvornum.se>
2021-12-16Changed lsblk to blkid where possible (#802)Anton Hvornum
* Swapped lsblk for blkid * Added a hefty sleep on partprobe() And added a TODO for the future
2021-12-12Fixing possible race condition with partprobe (#796)Wilhelm Åkermark
* Update partition.py * Added sleep after partprobe Added sleep after partprobe. Without this tools such as lsblk may be unable to retrieve certain properties of devices (such as partuuid).
2021-11-24Support encrypting multiple partitions (#759)Anton Hvornum
* Added support for storing disk encryption keyfiles and add them to a keyslot. * Added a luks2().add_key() function in order to inject a keyfile into a keyslot on a encrypted volume. * Simplified 'missing encryption password' logic in Filesystem(). Added a call to luks2().add_key() after the root-password is set on the volume, to add the keyfile in slot 2 * Adding in password handling in luks2().add_key(). It's required to enter a previous passphrase to unlock the volume and add a new keyslot. Also simplified the handling of partition in Installer().mount_ordered_layout() * Adding in encryption on all partitions except /boot when encryption is opted in * Removed setting size on Partition() as it's a read only value. No idea how Partition().size = size hasn't caused an issue before. Removed size=X argument to Partition() * Added a uniqueness to the loopdevice name. This should ensure that multiple encrypted volumes can be opened at the same time, except for Partition().detect_inner_filesystem() operations which can only happen one at a time since they share namespace. This should never be an issue since archinstall is single threaded and no concurrent operations can/should happen. * Added partprobe() as part of disk/helpers.py, added a /dev/ -> UUID mapper function called convert_device_to_uuid(path). Added a luks2().crypttab() function that sets up a /etc/crypttab entry. * Moved the responsability for telling archinstall to generate a keyfile from Filesystem() to user_interaction.py. This should in the future be a user-input based value, and not something the Filesystem() automatically dictates. * Added a retry mechanism to luks2().encrypt() to avoid having to re-start the installation when a device simply wasn't up yet. * Swapping UUID= lookup from loopdev to physdev.
2021-11-22Update filesystem.pyFirminator
just a few typos * label instead of labling * make UUID consistent
2021-11-18Increased disk delays by 100% (for unsuccessful operations), successful ↵Anton Hvornum
should still be quick. (#730) Co-authored-by: Anton Hvornum <anton.feeds@gmail.com>
2021-11-17Fixing broken encryption support in GRUB (#724)Anton Hvornum
* Added multiple `partprobe` calls and added a `.partprobe()` function on partitions, filesystem and blockdevice. * Adding retry attempts to all UUID related operations tied to the boot process * Tweaked logging for mounting and disk related operations * Removed potential SysCall exception disruptor causing exceptions to go by unnoticed * Increased the start position from 1MiB to 5MiB of /boot partition * Optimized the GRUB installation & config code * Improved Partition().uuid to never return None. Instead it will raise an exception if it can't get a PARTUUID within X retries with Y delay per attempt. * Increased sleep timer for partition uuid retrieval, because even with a 3 second sleep it wasn't long enough even on fast devices. * Make GRUB install to /dev/sda instead of /dev/sda1. * Added 10 retries for retreiving PARTUUID with a one second sleep. Instead of increasing the sleep simply add more retries until we find a good balance on slower disks.
2021-11-12Removed the GPT vs MBR lookup on __enter__ as it's no longer necessary to ↵Anton Hvornum
validate this on instance creation. load_layout() Uses this only to detect what partition table format it should use when wiping the drive. Other than that we only check if MBR and part numbers are > 3, that's the only use of this variable at this moment.
2021-11-11Added more debugging (some duplicate, but that's fine)Anton Hvornum
2021-11-11Adding in debuggingAnton Hvornum
2021-11-11elif -> ifAnton Hvornum
2021-11-11Removed legacy code related to disk wiping. Also added in partprobe in ↵Anton Hvornum
strategic places (not sure this is enough, will have to test)
2021-11-11Merged PR #707 - Fixes password leakage to terminalAnton Hvornum
Created a new JSON serializer called `UNSAFE_JSON` that will serialize everything, including sensitive information. And `JSON` which is the default up to this point now safely ignores any sensitive information in dictionaries marked with `!`, for instance `{"!password" : "mypassword"}` will be omitted from any output.
2021-11-10Solves issue #689. Attribute Error.'None type' has no attribute 'format' ...Werner Llácer
It seems the system does not syncronus update its internal information after a partitioning. Two places are affected. Directly on filesystem.add_partition (the uuid of the new partition isn't available after the parted command) and blockdevice.get_partition, where the list of partitions for the iterator might not be available in the query. The patch places both sections under controlled loops, giving the system the chance to update the information. Should be more controlled via application parameters
2021-11-09Changed partition password to !passwordAnton Hvornum
And also added a check to grab !encryption-password if no partition-specific password was given.
2021-11-09Adding a exception to filesystem.parted() if it fails. We can't let these go ↵Anton Hvornum
by unoticed
2021-11-08Fix user_interaction import as well - this is also in the parent directory.Dylan M. Taylor
2021-11-05Fixed some import errors that snuk in masterAnton Hvornum
2021-11-05Cleaned up all flake8 issues/warnings. Did some code cleaning as well, ↵Anton Hvornum
mostly how we called things in guided.py but also some SysCommand calls
2021-10-30Adding support for passing arguments to .format()Anton Hvornum
This should enable people to use custom option arguments in their config files when scripting installations or using the API.
2021-10-30Added creating and mounting of subvolume structure for BTRFS.Anton Hvornum
2021-10-22Forgot some imports that didn't show up on a static run without going ↵Anton Hvornum
through a few of the menu's