bzr-svn aborts instead of waiting if its sqlite DB is locked

You can work around this by installing python-tdb, but unfortunately that doesn't work on windows.

Can you detail or point out how to install python-tdb? I don't have root access.
I downloaded from http://tdb.samba.org and tried:
$ ./configure --prefix=${HOME}
$ make
$ make install

And tried again, still, bzr-svn uses sqlite3. Are there any environment variables to set or config variables to make bzr-svn use tdb?

On Tue, 2010-12-14 at 06:14 +0000, Yuval Hager wrote:
> Can you detail or point out how to install python-tdb? I don't have root access.
> I downloaded from http://tdb.samba.org and tried:
> $ ./configure --prefix=${HOME}
> $ make
> $ make install
>
> And tried again, still, bzr-svn uses sqlite3. Are there any environment
> variables to set or config variables to make bzr-svn use tdb?
The easiest thing to do is probably to install the python-tdb package
from your distribution.

Alternatively you can set LD_LIBRARY_PATH and PYTHONPATH appropriately -
LD_LIBRARY_PATH to the path holding libtdb.so.1 and PYTHONPATH to the
path with tdb.so.

Cheers,

Jelmer

Thanks, although I am giving up now. I am forced to use RHEL5.5, which has python2.4. I tried compiling tdb, but configure fails to find python-config.
I then tried to use ActiveState's python or python from source and install tdb on top of that (had to tweak the configure script a bit, to find the correct python version). Eventually, I always ended up with:
$ python -c 'import tdb'
Fatal Python error: Interpreter not initialized (version mismatch?)
Aborted

I straced it and verified that it loads the correct tdb library that I just compiled for the same python version, but nothing worked.

This is probably remotely related to bzr-svn, but I am writing this here, since this is what stops me from using it at the moment :-(

Running with
use-cache = False

in ~/.bazaar/bazaar.conf workarounds this problem.

I think that's just because calling sqlite didn't call commit after execute!

=== modified file cache/sqlitecache.py
--- cache/sqlitecache.py 2011-07-30 16:44:57 +0000
+++ cache/sqlitecache.py 2011-08-02 13:04:50 +0000
@@ -240,7 +240,11 @@
             orig_mapping_name = original_mapping.name
         else:
             orig_mapping_name = None
- self.cachedb.execute("insert into original_mapping (path, revnum, original_mapping) values (?, ?, ?)", (foreign_revid[1].decode("utf-8"), foreign_revid[2], orig_mapping_name))
+ try:
+ self.cachedb.execute("insert into original_mapping (path, revnum, original_mapping) values (?, ?, ?)", (foreign_revid[1].decode("utf-8"), foreign_revid[2], orig_mapping_name))
+ self.commit()
+ except:
+ raise Exception("set_original_mapping error with %s : %s!" % (foreign_revid, orig_mapping_name))

     def insert_revision(self, foreign_revid, mapping, (revno, revid, hidden),
             stored_lhs_parent_revid):

On Tue, 2011-08-02 at 13:08 +0000, Yonggang Luo wrote:
> I think that's just because calling sqlite didn't call commit after
> execute!

> === modified file cache/sqlitecache.py
> --- cache/sqlitecache.py 2011-07-30 16:44:57 +0000
> +++ cache/sqlitecache.py 2011-08-02 13:04:50 +0000
> @@ -240,7 +240,11 @@
> orig_mapping_name = original_mapping.name
> else:
> orig_mapping_name = None
> - self.cachedb.execute("insert into original_mapping (path, revnum, original_mapping) values (?, ?, ?)", (foreign_revid[1].decode("utf-8"), foreign_revid[2], orig_mapping_name))
> + try:
> + self.cachedb.execute("insert into original_mapping (path, revnum, original_mapping) values (?, ?, ?)", (foreign_revid[1].decode("utf-8"), foreign_revid[2], orig_mapping_name))
> + self.commit()
> + except:
> + raise Exception("set_original_mapping error with %s : %s!" % (foreign_revid, orig_mapping_name))
This doesn't help. It just means writing the data to the database more
often, which hurts performance. That might slightly decrease the window
where there is a lock open on the database, but it doesn't solve the
fundamental problem.

Doing commits at the wrong time can also make it becomes possible to
write incomplete data to the database, which can cause a corrupt cache
if bzr-svn gets killed at the wrong moment.

Cheers,

Jelmer

(CC'ing the bug. Usually, the bug tracker is the right place for
comments specific to a particular bug. )

On Wed, 2011-08-03 at 00:43 +0800, 罗勇刚(Yonggang Luo) wrote:
> Maybe this is the solution.

> http://stackoverflow.com/questions/5529820/sqlite3-operationalerror-database-is-locked

> We need specify separate sql database for each repo, not for each url.
That would negate a lot of the benefit of actually having a cache. It's
possible to have multiple processes working against the same bzr
repository - which would mean we could still hit locking issues.

We don't need a database that can support multiple writers with high
concurrency. It's fine if we have to wait for another process to release
the database lock - the only thing that it could be working on is
updating the cache, and that's what the process that's waiting would do
too.

The main thing that's missing is actually asking sqlite to wait for the
lock to be released rather than raising an exception immediately if it
finds a lock. IIUC this is possible, but I haven't had the time to look
up the right functions in the API, write some tests and patch bzr-svn to
use it.

Cheers,

Jelmer

>
> 2011/8/2 Jelmer Vernooij <email address hidden>
> On Tue, 2011-08-02 at 20:44 +0800, 罗勇刚(Yonggang Luo) wrote:
> > Delete br llvm/branches/release_21:41946
> > Delete br llvm/branches/release_21:41945
> > Delete br llvm/branches/release_21:41943
> > Delete br llvm/branches/release_21:41920
> > bzr: ERROR: sqlite3.OperationalError: database is locked
>
> Please see the bug tracker:
>
> This is http://pad.lv/685251
>
> Patches welcome.
>
> Cheers,
>
> Jelmer
>
>
>
>
>
> --
> 此致
> 礼
> 罗勇刚
> Yours
> sincerely,
> Yonggang Luo
>
>

Fixed in stable r13742 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/13742>.

This can't be QA'ed at the moment because tellarium appears to be down, which breaks code imports on qastaging.