Send patches - preferably formatted by git format-patch - to patches at archlinux32 dot org.
summaryrefslogtreecommitdiff
path: root/lib/libalpm/be_files.c
AgeCommit message (Collapse)Author
2010-10-13Complete rework of package accessor logicDan McGee
Hopefully we've finally arrived at package handling nirvana, or at least this commit will get us a heck of a lot closer. The former method of getting the depends list for a package was the following: 1. call alpm_pkg_get_depends() 2. this method would check if the package came from the cache 3. if so, ensure our cache level is correct, otherwise call db_load 4. finally return the depends list Why did this suck? Because getting the depends list from the package shouldn't care about whether the package was loaded from a file, from the 'package cache', or some other system which we can't even use because the damn thing is so complicated. It should just return the depends list. So what does this commit change? It adds a pointer to a struct of function pointers to every package for all of these 'package operations' as I've decided to call them (I know, sounds completely straightforward, right?). So now when we call an alpm_pkg_get-* function, we don't do any of the cache logic or anything else there- we let the actual backend handle it by delegating all work to the method at pkg->ops->get_depends. Now that be_package has achieved equal status with be_files, we can treat packages from these completely different load points differently. We know a package loaded from a zip file will have all of its fields populated, so we can set up all its accessor functions to be direct accessors. On the other hand, the packages loaded from the local and sync DBs are not always fully-loaded, so their accessor functions are routed through the same logic as before. Net result? More code. However, this code now make it roughly 52 times easier to open the door to something like a read-only tar.gz database backend. Are you still reading? I'm impressed. Looking at the patch will probably be clearer than this long-winded explanation. Signed-off-by: Dan McGee <dan@archlinux.org> [Allan: rebase and adjust] Signed-off-by: Allan McRae <allan@archlinux.org>
2010-10-13Allow local and sync db to be treated separatelyDan McGee
Implement this seemingly simple change in package.h: typedef enum _pmpkgfrom_t { - PKG_FROM_CACHE = 1, - PKG_FROM_FILE + PKG_FROM_FILE = 1, + PKG_FROM_LOCALDB, + PKG_FROM_SYNCDB } pmpkgfrom_t; which requires flushing out several assumptions from around the codebase with regards to usage of the PKG_FROM_CACHE value. Make some changes where required to allow the switch, and now the correct value should be set (via a crude hack) depending on whether a package was loaded as an entry in a local db or a sync db. This patch underwent some big rebasing from Allan and Dan. Signed-off-by: Dan McGee <dan@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2010-10-11Avoid stat call to determine is_directory if possibleDan McGee
On Linux and OS X, we can determine if an entry obtained through a readdir() call is a directory without also having to stat it. This can save a significant number of syscalls. The performance increase isn't dramatic, but it could be on some platforms (e.g. Cygwin) so it shouldn't hurt to use this unconditionally where supported. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-08-23fgets invocation cleanupDan McGee
From the fgets manpage: fgets() reads in at most one less than size characters from stream and stores them into the buffer pointed to by s. Reading stops after an EOF or a newline. If a newline is read, it is stored into the buffer. A '\0' is stored after the last character in the buffer. This means there is no need at all to do 'size - 1' math. Remove all of that and just use sizeof() for simplicity on the buffer we plan on reading into. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-07-07Download sync db into DBPath/syncAllan McRae
The sync db should be stored in the sync/ folder. This cleans up DBPath to only have local/ and sync/ directories in it. A nice side effect is that the db are now in the right place so we can implement directly reading from them. Signed-off-by: Allan McRae <allan@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2010-07-07Remove DBEXT usageAllan McRae
With commit 5dffef78, the repo database always has a symlink of the form reponame.db. Use that filename and let libarchive determine the compression type. Signed-off-by: Allan McRae <allan@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2010-07-07Check return value of fgets callsAllan McRae
Prevents compiler warnings with -D_FORTIFY_SOURCE=2 Signed-off-by: Allan McRae <allan@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2010-03-25More consistent printing of off_t and time_tXavier Chantry
time_t : %ld off_t : %jd and cast to intmax_t Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2010-03-15Unbreak the database partial extraction codeDan McGee
Basically I'm the idiot that thought I could make it better and completely forgot how freeing the contents of the original lists would screw up our nice little diff extraction lists. This caused segfaults among other problems. Last time I try to do that... Program received signal SIGSEGV, Segmentation fault. 0x00007ffff627ce26 in strcmp () from /lib/libc.so.6 (gdb) bt Signed-off-by: Dan McGee <dan@archlinux.org>
2010-03-14fix a few warnings reported by clangXavier Chantry
- remove unused variables - some more sanity checks - safer printf Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2010-03-14Reduce duplicate code in DB extractionDan McGee
Follow-up to the previous "Only extract new DB entries" patch; move the partial extraction code inside one side of the loop so we can use the same code for actually doing file extraction. Signed-off-by: Dan McGee <dan@archlinux.org>
2010-03-14Only extract new DB entriesXavier Chantry
This implements FS#15198. The idea apparently came from Csaba Henk <csaba-ml <at> creo.hu> which submitted a patch to Frugalware, so thanks to him, even though I did not look at the code :) The idea is to only extract folders for new packages into the package database and clean up the old directories. This is essentially implementing Xyne's "rebase" script within pacman. If using -Syy, just remove and extract everything. If using -Sy : 1. Generate list of directories in DB 2. Generate list of directories in archive 3. Compare both 4. Clean up old directories 5. Extract new directories Original-work-by: Allan McRae <allan@archlinux.org> Signed-off-by: Xavier Chantry <shiningxc@gmail.com> [Dan: fix compile error, s/int/size_t/] Signed-off-by: Dan McGee <dan@archlinux.org>
2009-12-13Print error on duplicated database entriesNagy Gabor
Some users reported duplicated database entries in /var/lib/pacman/local/, for example, both foo-1.0-1 and foo-2.0-1 subdirectories existed. (Bogus 3rd-party scripts, backup?) In this case pacman reported no error and its behaviour was mysterious. From now on, pacman detects this situation and prints an error message. Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu> Signed-off-by: Dan McGee <dan@archlinux.org>
2009-11-15download: major refactor to address lingering issuesDan McGee
Sorry for this being such a huge patch, but I believe it is necessary for quite a few reasons which I will attempt to explain herein. I've been mulling this over for a while, but wasn't super happy with making the download interface more complex. Instead, if we carefully order things in the internal download code, we can actually make the interface simpler. 1. FS#15657 - This involves `name.db.tar.gz.part` files being left around the filesystem, and then causing all sorts of issues when someone attempts to rerun the operation they canceled. We need to ensure that if we resume a download, we are resuming it on exactly the same file; if we cannot be almost postive of that then we need to start over. 2. http://www.mail-archive.com/pacman-dev@archlinux.org/msg03536.html - Here we have a lighttpd bug to ruin the day. If we send both a Range: header and If-Modified-Since: header across the wire in a GET request, lighttpd doesn't do what we want in several cases. If the file hadn't been modified, it returns a '304 Not Modified' instead of a '206 Partial Content'. We need to do a stat (e.g. HEAD in HTTP terms) operation here, and the proceed accordingly based off the values we get back from it. 3. The mtime stuff was rather ugly, and relied on the called function to write back to a passed in reference, which isn't the greatest. Instead, use the power of the filesystem to contain this info. Every file downloaded internally is now carefully timestamped with the remote file time. This should allow the resume logic to work. In order to guarantee this, we need to implement a signal handler that catches interrupts, notifies the running code, and causes it to set the mtimes on the file. It then rethrows the signal so the pacman signal handler (or any frontend) works as expected. 4. We did a lot of funky stuff in trying to track the DB last modified time. It is a lot easier to just keep the downloaded DB file around and track the time on that rather than in a funky dot file. It also kills a lot of code. 5. For GPG verification of the databases down the road, we are going to need the DB file around for at least a short bit of time anyway, so this gets us closer to that. Signed-off-by: Dan McGee <dan@archlinux.org> [Xav: fixed printf with off_t] Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
2009-10-19Size handling was changed in fgets() functionsLaszlo Papp
Pacman's fgets function in the API used hardcoded numbers to identify the size. This is not good practice, so replace them with sizeof handling. Signed-off-by: Laszlo Papp <djszapi@archlinux.us> Signed-off-by: Dan McGee <dan@archlinux.org>
2009-10-11Rework the alpm_unpack functionsXavier Chantry
Add support to extract a list of entries Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2009-10-11cygwin fix : use unsigned char for ctype functionXavier Chantry
See http://www.nabble.com/-PATCH-RFA--Distinguish-between-EOF-and-character-with-value-0xff-td23161772.html#a23188494 cygwin 1.7 actually displays a warning when using signed char with the ctype function, so that compilation fails when using -Wall -Werror. So we just cast all arguments to unsigned char. Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2009-09-20Fully implement database lazy loadingDan McGee
Commit 34e1413d75 attempted to implement lazy loading of package databases. Although it took care of my main complaint (creating the database directory if it didn't exist), it didn't allow sync repos to be registered before alpm_option_set_dbpath() had been called. With this patch, we no longer compute the individual repository DB paths until necessary, allowing full lazy loading to work as intended, and allowing us to drop the extra setlibpath() calls from the frontend. This allows the changes introduced in a2cd48960 (but later reverted) to be added back in again. Signed-off-by: Dan McGee <dan@archlinux.org>
2009-09-20Merge branch 'maint'Dan McGee
Message updates made this one a bit messy, but nothing too bad. Conflicts: lib/libalpm/add.c lib/libalpm/remove.c
2009-09-16String improvementsXavier Chantry
Add more untranslated strings, improve consistency, etc. Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2009-09-08Remove transaction typeXavier Chantry
This basically started with this change : /* Transaction */ struct __pmtrans_t { - pmtranstype_t type; pmtransflag_t flags; pmtransstate_t state; - alpm_list_t *packages; /* list of (pmpkg_t *) */ + alpm_list_t *add; /* list of (pmpkg_t *) */ + alpm_list_t *remove; /* list of (pmpkg_t *) */ And then I have to modify all the code accordingly.
2009-05-11Fix non-C style comments in libalpmGerardo Exequiel Pozzi
Signed-off-by: Gerardo Exequiel Pozzi <vmlinuz386@yahoo.com.ar> Signed-off-by: Dan McGee <dan@archlinux.org>
2009-04-11Add detail documentation for alpm_db_update()Sebastian Nowicki
This adds alpm_db_update() to the alpm_databases Doxygen group. The function is described in more detail and a code example is given. Signed-off-by: Dan McGee <dan@archlinux.org>
2009-03-15be_files.c : use %jd instead of %ju for intmax_tXavier Chantry
Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2009-01-20Delay the creation of local and sync db dir.Xavier Chantry
We don't need to create the directories when local or sync dbs are registered. For example, if a sync db does not exist, we cannot even do "pacman -Q" as an user. Instead, we can create the local db if needed during the db_prepare operation, and sync dbs on db_update. Also remove some more useless abstractions in db_update and switch to a much more efficient way to remove a sync db : rm -rf. Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
2009-01-20Remove some db abstraction crap.Xavier Chantry
These db_open and db_close looked quite useless. And they caused the db directory to be opened on a simple registering of a database. This is totally unneeded, this opening can be delayed to when we actually need it. Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
2009-01-12Rename alpm_get_md5sum to alpm_compute_md5sum and alpm_dep_get_string to ↵Nagy Gabor
alpm_dep_compute_string This patch introduces the following function name convention: _compute_ in function name: the return value must be freed. _get_ in function name: the return value must not be freed. Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu> Signed-off-by: Dan McGee <dan@archlinux.org>
2009-01-02Separate local db directory creation and db writeAllan McRae
Changelogs and install files were getting extracted into the local db folder before it was manually created. This created issues for uses with 0077 umasks and was highlighted with the new sudo handling of umasks (FS#12263). This moves the local db creation to its own function which is called before the start of package archive extraction. Also, added a check that the folder is actually created. Signed-off-by: Allan McRae <allan@archlinux.org> [Dan: rename to _alpm_db_prepare()] Signed-off-by: Dan McGee <dan@archlinux.org>
2008-11-30Add regex to delta code so we don't segfault when reading lineDan McGee
If the delta line doesn't match our regex, we won't go and process it, possibly walking off the end of the string. Signed-off-by: Dan McGee <dan@archlinux.org>
2008-11-17Minor code cleanupsDan McGee
Mostly noticed when compiling libalpm/pacman with ICC. Signed-off-by: Dan McGee <dan@archlinux.org>
2008-06-15Use access() instead of stat() when possibleDan McGee
We were using the stat() system call in quite a few places when we didn't actually need anything the stat struct returned- we were simply checking for file existence. access() will be more efficient in those cases. Before (strace pacman -Ss pacman): % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 33.16 0.005987 0 19016 stat64 After: % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 34.85 0.003863 0 12633 1 access 7.95 0.000881 0 6391 7 stat64 Signed-off-by: Dan McGee <dan@archlinux.org>
2008-06-04Get rid of double / in database paths.Xavier Chantry
Errors like the following one happen regularly (for unknown reasons...) : error: could not open file /var/lib/pacman/local//glibc-2.7-9/depends: No such file or directory Anyway, every time an user reported an error like that, it always seemed like he thought the error was caused by the double /, which is obviously wrong. Since db->path always include a trailing /, there is no need to add one when concatenating paths in be_files.c or add.c. Additionally, some static strings were switched to dynamic. And the computation of the "dbpath"/"pkgname"-"pkgversion" was refactored in db_read, db_write and db_remove with a get_pkgpath static function. Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2008-06-04Use correct C type for file sizesDan McGee
We have been using unsigned long as a file size type for a while, which works but isn't quite correct and could easily break. Worse was probably our use of int in the download callback functions, which could be restrictive for packages > 2GB in size. Switch all file size variables to use off_t, which is the preferred type for file sizes. Note that at least on Linux, all applications compiled against libalpm must now be sure to use large file support, where _FILE_OFFSET_BITS is defined to be 64 or there will be some weird issues that crop up. Signed-off-by: Dan McGee <dan@archlinux.org>
2008-05-29be_files : stores REPLACES and FORCE in desc.Xavier Chantry
repo-add and db_read both assume that REPLACES and FORCE fields are in the desc file, so do that for db_write as well (instead of depends file). Note that db_write is currently only used on the local database. And the only purpose of replaces and force in local database is for information purpose (available on -Qi operations). So this is not a big problem. Ref: http://www.archlinux.org/pipermail/pacman-dev/2008-May/011859.html Acked-by: Dan McGee <dan@archlinux.org> Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2008-05-29Add some verbosity to _alpm_db_read.Nagy Gabor
Name and version are computed from "/var/lib/pacman/..." pathname. And the %NAME% and %VERSION% fields from the desc file were not even read. So now, when we read the desc file, we make sure the %NAME% and %VERSION% fields are consistent. Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu> Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2008-05-13Simplify _alpm_pkg_new()Dan McGee
Any real call of this function doesn't specify a name or version ahead of time, so just kill that functionality off. Now to remove those dummy packages... Signed-off-by: Dan McGee <dan@archlinux.org>
2008-05-13Fix rewinddir regression by cleaning up db_scanXavier Chantry
Commit 046003844739416ff6d168dd2dec76490adb0727 caused a regression when rereading the pkgcache after updating the on-disk databases. A rewinddir call was errantly removed. Instead of replacing the call to rewindir, clean up this whole mess. db_scan is used only once and with target == NULL so there was actually half the code of db_scan which was unused. This is gone now and replaced by a single new db_populate function. Dan: add_sorted ended up being 3x slower than one msort at the end, so I changed back to that. I also made one pointer variable const and merged this whole patch with my original fix for the rewinddir issue. Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2008-05-11Remove some useless abstraction and start db cleanupDan McGee
We have some useless abstractions like an alpm_db_rewind function. I've read somewhere that readdir() was the worst filesystem function call invented, and what do we do? Add a wrapper around it. Kill this abstraction and move some other things into be_files that should be there anyway because they are so tied to how a files backend works. Signed-off-by: Dan McGee <dan@archlinux.org>
2008-05-08be_files.c: PATH_MAX cleanupDan McGee
Most of these are not easy to remove, but I could kill the ones in the two lastupdate functions. Signed-off-by: Dan McGee <dan@archlinux.org>
2008-04-06Remove unnecessary header file, move one macro to util.cDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2008-01-21Move the deptest code from frontend to backend.Chantry Xavier
The deptest code (pacman -T) used by makepkg was mostly in the frontend. There were 2 drawbacks: 1) the public splitdep function returns a pmdepend_t struct, but the _alpm_dep_free function for freeing it is private. So there was a memleak. 2) there is a helper in the backend (satisfycmp in deps.c) which makes this function much easier. So this adds a new public alpm_deptest in libalpm/deps.c, which cleans pacman_deptest in pacman/deptest.c a lot. Besides, alpm_splitdep was made private, because the frontend no longer requires it, and _alpm_dep_free is also private. Finally the deptest001 pactest was extended. Signed-off-by: Chantry Xavier <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2008-01-21Convert deltas to dynamic allocationDan McGee
Another elimination of a static length structure in libalpm. Should result in a little more memory saved during execution of packages with lots of deltas attached. Signed-off-by: Dan McGee <dan@archlinux.org>
2008-01-12Store replaces in the local DB tooAaron Griffin
Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com> [Dan: might as well store 'force' too] Signed-off-by: Dan McGee <dan@archlinux.org>
2008-01-11Use dynamic string allocation in package structuresDan McGee
This also affects all structures with static strings, such as depmiss, conflict, etc. This should help a lot with memory usage, and hopefully make things a bit more "idiot proof". Currently our pactest pass/fail rate is identical before and after this patch. This is not to say it is a perfect patch- I have yet to pull valgrind out. However, this should be quite safe to use in all situations from here on out, and we can start plugging the memleaks. Original-work-by: Aaron Griffin <aaronmgriffin@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
2008-01-01Remove gettext calls from debug-level messagesDan McGee
These used FUNCTION output level and not DEBUG, so I didn't catch them way back when I removed those gettext calls. Remove them now (which exposed a nice little memory access error elsewhere in the code). This should have a slight speedup effect on the code too as we no longer have to make the gettext call even when these messages aren't printed. Signed-off-by: Dan McGee <dan@archlinux.org>
2007-12-10Update GNU GPL boilerplate and copyright datesDan McGee
Update the GPL boilerplate to direct people to the GNU website for a copy of the license, as well as bump all of Judd's copyrights to 2007. Signed-off-by: Dan McGee <dan@archlinux.org>
2007-11-20Add missing header includes for setlocaleDan McGee
Signed-off-by: Dan McGee <dan@archlinux.org>
2007-11-16War on whitespaceDan McGee
Run the kernel's cleanfile script on all of our source files. Signed-off-by: Dan McGee <dan@archlinux.org>
2007-11-16libalpm: simplify sync db lastupdateDan McGee
Legacy code is hitting the trash here. Remove unnecessary _alpm_time2string time storage abstraction in favor of just writing the time_t value to the disk. The only drawback is that everyone's sync DBs will have to be updated at least once so that the lastupdate values are stored right. :) Signed-off-by: Dan McGee <dan@archlinux.org>
2007-11-14Move alpm_splitdep usage to db_readDan McGee
Holy inefficient batman! For a pacman -Qt operation (when we are using compute_requiredby and not database entries), splitdep was being called ~1.3 million times on my local database. By splitting when we read the DB, we drop this number to around 1700 and save a LOT of time in doing so (a 5x increase in pacman -Qt speed here). Note that the depends alpm_list_t in the package struct is no longer a string list, but a list of pmdepent_t objects. Signed-off-by: Dan McGee <dan@archlinux.org>