Send patches - preferably formatted by git format-patch - to patches at archlinux32 dot org.
summaryrefslogtreecommitdiff
path: root/lib/libalpm/sync.c
AgeCommit message (Collapse)Author
2011-10-25lib/sync: inform callers of compute_download_size of a partialDave Reisner
Extend the return values of compute_download_size to allow callers to know that a .part file exists for the package. This extra value isn't currently used, but it'll be needed later on. Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-10-21sync: move file download loop out of download_filesDave Reisner
Create a new static function called 'download_single_file' which iterates over the servers for each payload. Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-10-21sync: check for necessary disk space for downloadDave Reisner
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
2011-10-21sync: extract build_payload() method from find_dl_candidatesDan McGee
This is done by both the delta and regular file code, so we can extract a little helper method. Done mostly to satisfy my "why are we repeating code here" itch. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-10-21sync: dont group sync records by repositoryDave Reisner
Break out the logic of finding payloads into a separate static function to avoid nesting mayhem. After gathering all the records, download them all at once. Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-10-17dload: add pointer to server list for each payloadDave Reisner
Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-10-12Convert MALLOC to actually call malloc()Dan McGee
If you need zero-filled allocations, call CALLOC() instead. This was from the original definition of these macros in commit cc754bc6e3be0f3; hopefully our code is in the shape it needs to be to switch this behavior. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-28Refactor download payload reset and freeDan McGee
This was done to squash a memory leak in the sync database download code. When we downloaded a database and then reused the payload struct, we could find ourselves calling get_fullpath() for the signatures and overwriting non-freed values we had left over from the database download. Refactor the payload_free function into a payload_reset function that we can call that does NOT free the payload itself, so we can reuse payload structs. This also allows us to move the payload to the stack in some call paths, relieving us of the need to alloc space. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-28Track unresolvable transaction packagesDan McGee
Rather than free them right away, keep the list on the transaction as we already do with add and remove lists. This is necessary because we may be manipulating pointers the frontend needs to refer to packages, and we are breaking our contract as stated in the alpm_add_pkg() documentation of only freeing packages at the end of a transaction. This fixes an issue found when refactoring the package list display code. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-28Fix memory leak in download payload->remote_nameDan McGee
In the sync code, we explicitly allocated a string for this field, while in the dload code itself it was filled in with a pointer to another string. This led to a memory leak in the sync download case. Make remote_name non-const and always explicitly allocate it. This patch ensures this as well as uses malloc + snprintf (rather than calloc) in several codepaths, and eliminates the only use of PATH_MAX in the download code. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-27Delay check for repository servers until we need themDan McGee
In the sync download code, we added an early check in 6731d0a9407e for sync download server existence so we wouldn't show the same error over and over for each file to be downloaded. Move this check into the download block so we only run it if there are actually files that need to be downloaded for this repository. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-22Update Doxyfile and fix some documentation errors caught by DoxygenDan McGee
A few parameters were outdated or wrongly named, and a few things were explicitly linked that Doxygen wasn't able to resolve. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-22Add _alpm_process_siglist() logic to failed package validationDan McGee
This moves the result processing out of the validation check loop itself and into a new loop. Errors will be presented to the user one-by-one after we fully complete the validation loop, so they no longer overlap the progress bar. Unlike the database validation, we may have several errors to process in sequence here, so we use a function-scoped struct to track all the necessary information between seeing an error and asking the user about it. The older prompt_to_delete() callback logic is still kept, but only for checksum failures. It is debatable whether we should do this at all or just delegate said actions to the user. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-22validate_deltas: split verify/check errors loopsDan McGee
This allows us to do all delta verification up front, followed by whatever needs to be done with any found errors. In this case, we call prompt_to_delete() for each error. Add back the missing EVENT(ALPM_EVENT_DELTA_INTEGRITY_DONE) that accidentally got removed in commit 062c391919e93f1d6. Remove use of *data; we never even look at the stuff in this array for the error code we were returning and this would be much better handled by one callback per error anyway, or at least some strongly typed return values. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-20_alpm_sync_commit: extract two static methodsDan McGee
This adds two new static methods, check_validity() and load_packages(), to sync.c which are simply code fragments pulled out of our do-everything sync commit code. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-20Pass package signature data up one more levelDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-20Split package validation and load loopsDan McGee
This adds a some new callback event and progress codes for package loading, which was formerly bundled in with package validation before. The main sync.c loop where loading occurred is now two loops running sequentially. The behavior should not change with this patch outside of progress and event display; more changes will come in following patches. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-20Extract an _alpm_pkg_validate_internal() methodDan McGee
_alpm_pkg_load_internal() was becoming a monster. Extract the top bit of the method that dealt with checksum and signature validation into a separate method that should be called before one loads a package to ensure it is valid. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-20Remove all usages of alpm_list_getdata() from the libraryDan McGee
No need for the indirection; just access ->data instead. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-19Be smarter about running ldconfig during removal transactionsDan McGee
1. Don't run it if something failed in package removal- this mirrors what we already do in sync transactions. 2. Don't run it if we are invoking it for the replaces removal bit of a sync transaction- it doesn't make sense to run ldconfig halfway through a sync install; we should only run it once at the end. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-11make compute_download_size consider .part filesDave Reisner
Check for the existance of a partial download of a package file before jumping to delta calculations. Currently, if there were 10MiB remaining in a 100MiB the values passed to the front end do not reflect this. Refactored from an old patch originally by Dan. Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-02More package operations cleanupDan McGee
Neither deltas nor filename attributes are ever present in the local database, so we can remove all of the indirection for accessing these attributes. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-02Former transaction callback rename refactorDan McGee
Put all the callback stuff in alpm.h in one spot, and make the following renames for clarity with the new structure: ALPM_TRANS_EVT_* --> ALPM_EVENT_* ALPM_TRANS_CONV_* --> ALPM_QUESTION_* ALPM_TRANS_PROGRESS_* --> ALPM_PROGRESS_* alpm_option_get_convcb() --> alpm_option_get_questioncb() alpm_option_set_convcb() --> alpm_option_set_questioncb() Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-02Move all callbacks up to the handle levelDan McGee
This was just disgusting before, unnecessary to limit these to only usage in a transaction. Still a lot of more room for cleanup but we'll start by attaching them to the handle rather than the transaction we may or may not even want to use these callbacks. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-02Make delta validation/application more logicalDan McGee
The call to apply was tucked inside validate, and the EVENT callbacks were done outside the function rather than inside. Reorganize things a bit to make more sense. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-02Clean up handling of size fieldsDan McGee
We currently have csize, isize, and size concepts, and sometimes the difference isn't clear. Ensure the following holds: * size (aka csize): always the compressed size of the package; available for everything except local packages (where it will return 0) * isize: always the installed size of the package; available for all three package types Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-30Scale package integrity progress bar/percentage by package sizeDan McGee
This upgrades the simple 15/17 scaling by package number we used before to package sized based scaling, which is much more accurate. Addresses some of the issues raised in FS#25817. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-28Refactor some args out of pkg_load_internalDan McGee
Just pass the entire sync package in if we have it; that way we can do any necessary operations involving it rather than have a parameter list growing endlessly. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-23sync: halt file discovery if repo has no serversDave Reisner
This avoids error spam when no servers are configured for a repo and a sync operation is performed: Proceed with installation? [Y/n] y :: Retrieving packages from testing... warning: failed to retrieve some files from testing warning: failed to retrieve some files from testing warning: failed to retrieve some files from testing warning: failed to retrieve some files from testing warning: failed to retrieve some files from testing warning: failed to retrieve some files from testing warning: failed to retrieve some files from testing warning: failed to retrieve some files from testing warning: failed to retrieve some files from testing warning: failed to retrieve some files from testing Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-19dload: rename payload->filename to payload->remote_nameDave Reisner
This is a far more accurate description of what this is, since it's more than likely not really a filename at all, but the name after a final slash on a URL. Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-19Be more robust when copying package dataDan McGee
This changes the signature of _alpm_pkg_dup() to return an integer error code and provide the new package in a passed pointer argument. All callers are now more robust with checking the return value of this function to ensure a fatal error did not occur. We allow load failures to proceed as otherwise we have a chicken and egg problem- if a 'desc' local database entry is missing, the best way of restoring said file is `pacman -Sf --dbonly packagename`. This patch fixes a segfault that was occurring in this case. Fixes the segfault reported in FS#25667. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-19sync: fix garbled output in conflict promptDave Reisner
$ pacman -S cronie resolving dependencies... looking for inter-conflicts... :: cronie and fcron are in conflict (@.). Remove fcron? [y/N] n $ sudo pacman -S pacman resolving dependencies... looking for inter-conflicts... :: pacman and pacman-git are in conflict (pKÈ). Remove pacman-git? [y/N] Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-18Fix replacement of provider issueDan McGee
When we switched to using alpm_depcmp() in resolving replacments, we had some interesting behavior with regard to providers and packages not found in repositories. Teach the replacement resolving code to not look at provisions at all to be slightly more sane. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-18Replacements refactor: extract check_replacers()Dan McGee
This moves code that was inline in alpm_sync_sysupgrade() to its own method. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-18Replacements refactor: extract check_literal()Dan McGee
This moves code that was inline in alpm_sync_sysupgrade() to its own method. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-18Remove use of no-op accessor functions in libraryDan McGee
The functions alpm_db_get_name(), alpm_pkg_get_name(), and alpm_pkg_get_version() are not necessary at all, so remove the calling and indirection when used in the backend, which makes things slightly more efficient and reduces code size. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-18Enhance and utilize database status flagsDan McGee
* Move is_local standalone field to status enum * Create VALID/INVALID flag pair * Create EXISTS/MISSING flag pair With these additional fields, we can be more intelligent with database loading and messages to the user. We now only warn once if a sync database does not exist and do not continue to try to load it once we have marked it as missing. The reason for the flags existing in pairs is so the unknown case can be represented. There should never be a time when both flags in the same group are true, but if they are both false, it represents the unknown case. Care is taken to always manipulate both flags at the same time. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-16dload: zero out pm_errno in curl_download_internalDave Reisner
This reverts some hacky behavior from 5fc3ec and resets the handle's pm_errno where it should be reset -- prior to each download. This prevents a transaction with a download from being aborted when a package is successfully grabbed from a secondary server. Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-15lib/sync: ignore download errors for as long as possibleDave Reisner
Previously, the behavior was such that if a sync operation required packages from multiple repos, a download error in the first repo would cause a hard repo, ignoring the remainder of the repositories. Change this behavior so that we do a better job of fetching as many packages as possible before aborting the transaction. There's a little bit of refactoring mixed in here to get rid of some useless variables. Since we now depend heavily on the value of handle->pm_errno being accurate the determine the function's return value, we clear it when the transaction state is set. Fixes FS#25532. Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-15Further fixes to replacement iterationDan McGee
A partial fix for this was in commit 7de92cb22, but this should fix the remaining cases. There are still several issues dealing with "provision as replacement" selection however. Addresses FS#25538 and FS#25527. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-15Parse conflicts/provides/replaces at database load timeDan McGee
We did this with depends way back in commit c244cfecf654d3 in 2007. We can do it with these fields as well. Of note is the inclusion of provides even though only '=' is supported- we'll parse other things, but no guarantees are given as to behavior, which is more or less similar to before since we only looked for the equals sign. Also of note is the non-inclusion of optdepends; this will likely be resolved down the road. The biggest benefactors of this change will be the resolving code that formerly had to parse and reparse several of these fields; it only happens once now at load time. This does lead to the disadvantage that we will now always be parsing this information up front even if we never need it in the split form, but as these are uncommon fields and our parser is quite efficient it shouldn't be a big concern. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-15Validate the sha256sum if availableDan McGee
Adjust load_internal() to check the sha256sum value if we have it. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-15lib/sync: reset flag after rejecting a replaceDave Reisner
This prevents iteration through the remainder of the current tree, with pacman claiming that they're all replacements to the original replacement candidate. :: Synchronizing package databases... allanbrokeit is up to date testing is up to date core is up to date extra is up to date community-testing is up to date community is up to date :: Starting full system upgrade... :: Replace util-linux-git with core/util-linux? [Y/n] n :: Replace util-linux-git with core/vi? [Y/n] n :: Replace util-linux-git with core/vpnc? [Y/n] n :: Replace util-linux-git with core/wget? [Y/n] n :: Replace util-linux-git with core/which? [Y/n] n :: Replace util-linux-git with core/wireless-regdb? [Y/n] n :: Replace util-linux-git with core/wireless_tools? [Y/n] n :: Replace util-linux-git with core/wpa_actiond? [Y/n] n :: Replace util-linux-git with core/wpa_supplicant? [Y/n] n :: Replace util-linux-git with core/xfsprogs? [Y/n] n :: Replace util-linux-git with core/xinetd? [Y/n] n :: Replace util-linux-git with core/xz? [Y/n] n :: Replace util-linux-git with core/zd1211-firmware? [Y/n] n :: Replace util-linux-git with core/zlib? [Y/n] n there is nothing to do Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-09Parse replaces strings as dep strings with version specsDan McGee
This is done extremely crudely and is not very efficient, but it does push us down the path of being closer to right, as one additional test now passes. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-08Add reason to corrupted package callbackDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-08Attempt to fix up some of the brokenness around failed package loadsDan McGee
This is a bit of a mess, due to the fact that we have a progress meter running. It is also ironic that we are in the midst of a method named "commit" when we haven't done a damn thing yet, and can still fail hard if either a checksum or signature is invalid or unrecognized. Adapt the former test_md5sum method to be invoked for any of the various failure types, which at least gives the user some indication of what packages are failing. A second patch will be needed to actually show worthwhile error codes, but this is going to involve modifying the actual data passed with the callback. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-07-05absorb some _alpm_download params into payload structDave Reisner
Restore some sanity to the number of arguments passed to _alpm_download and curl_download_internal. Signed-off-by: Dave Reisner <dreisner@archlinux.org>
2011-07-05lib/dload: prevent large file attacksDave Reisner
This means creating a new struct which can pass more descriptive data from the back end sync functions to the downloader. In particular, we're interested in the download size read from the sync DB. When the remote server reports a size larger than this (via a content-length header), abort the transfer. In cases where the size is unknown, we set a hard upper limit of: * 25MiB for a sync DB * 16KiB for a signature For reference, 25MiB is more than twice the size of all of the current binary repos (with files) combined, and 16KiB is a truly gargantuan signature. Signed-off-by: Dave Reisner <dreisner@archlinux.org>
2011-07-05dload: handle irregular URLsDave Reisner
URLs might end with a slash and follow redirects, or could be a generated by a script such as /getpkg.php?id=12345. In both cases, we may have a better filename that we can write to, taken from either content-disposition header, or the effective URL. Specific to the first case, we write to a temporary file of the format 'alpmtmp.XXXXXX', where XXXXXX is randomized by mkstemp(3). Since this is a randomly generated file, we cannot support resuming and the file is unlinked in the event of an interrupt. We also run into the possibility of changing out the filename from under alpm on a -U operation, so callers of _alpm_download can optionally pass a pointer to a *char to be filled in by curl_download_internal with the actual filename we wrote to. Any sync operation will pass a NULL pointer here, as we rely on specific names for packages from a mirror. Fixes FS#22645. Signed-off-by: Dave Reisner <d@falconindy.com>
2011-07-05Allow invalid sync DBs to be returned by the libraryDan McGee
They are placeholders, but important for things like trying to re-sync a database missing a signature. By using the alpm_db_validity() method at the right time, a client can take the appropriate action with these invalid databases as necessary. In pacman's case, we disallow just about anything that involves looking at a sync database outside of an '-Sy' operation (although we do check the validity immediately after). A few operations are still permitted- '-Q' ops that don't touch sync databases as well as '-R'. Signed-off-by: Dan McGee <dan@archlinux.org>

Copyright © 2002-2022 Judd Vinet and Aaron Griffin. Copyright © 2022 Erich Eckner.
The Arch Linux name and logo are recognized trademarks. Some rights reserved. The registered trademark Linux® is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis.