bitbake/fetcher: Deal with a ton of different bugs

The more we try and patch up the fetcher code, the more things break. The
code blocks in question are practically unreadable and are full of corner
cases where fetching could fail. In summary the issues noticed included:

a) Always fetching strange broken urls from the premirror for "noclone"
   git repositories
b) Not creating or rewriting .md5 stamp files inconsistently
c) Always fetching git source mirror tarballs from the premirror even
   if they already exist but the checkout directory does now
d) Passing "None" values to os.access() and os.path.extsts() checks under
   certain circumstances
e) Not using fetched git mirror tarballs if the preexist and always
   try and fetch them.

This patch rewrites the sections of code in question to be simpler and
more readable, fixing the above problems and most likely other odd
corner cases.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

1 files changed, 16 insertions, 30 deletions

diff --git a/bitbake/lib/bb/fetch/__init__.py b/bitbake/lib/bb/fetch/__init__.py
index 55ffdb84c9..5fcb9b83e2 100644
--- a/bitbake/lib/bb/fetch/__init__.py
+++ b/bitbake/lib/bb/fetch/__init__.py
@@ -237,32 +237,14 @@ def go(d, urls = None):
     for u in urls:
         ud = urldata[u]
         m = ud.method
-        premirror_fetch = True
         localpath = ""
 
-        if ud.localfile:
-            if not m.try_premirror(u, ud, d):
-                premirror_fetch = False
-                # File already present along with md5 stamp file
-                # Touch md5 file to show activity
-                try:
-                    os.utime(ud.md5, None)
-                except:
-                    # Errors aren't fatal here
-                    pass
-
-            lf = bb.utils.lockfile(ud.lockfile)
-            if not m.try_premirror(u, ud, d):
-                premirror_fetch = False
-                # If someone else fetched this before we got the lock,
-                # notice and don't try again
-                try:
-                    os.utime(ud.md5, None)
-                except:
-                    # Errors aren't fatal here
-                    pass
+        if not ud.localfile:
+            continue
 
-        if premirror_fetch:
+        lf = bb.utils.lockfile(ud.lockfile)
+
+        if m.try_premirror(u, ud, d):
             # First try fetching uri, u, from PREMIRRORS
             mirrors = mirror_from_string(bb.data.getVar('PREMIRRORS', d, True))
             localpath = try_mirrors(d, u, mirrors, False, m.forcefetch(u, ud, d))
@@ -282,14 +264,18 @@ def go(d, urls = None):
                 if not localpath or not os.path.exists(localpath):
                     raise FetchError("Unable to fetch URL %s from any source." % u)
 
-        if localpath:
-            ud.localpath = localpath
-
-        if ud.localfile:
-            if not m.forcefetch(u, ud, d):
-                Fetch.write_md5sum(u, ud, d)
-            bb.utils.unlockfile(lf)
+        ud.localpath = localpath
+        if os.path.exists(ud.md5):
+            # Touch the md5 file to show active use of the download
+            try:
+                os.utime(ud.md5, None)
+            except:
+                # Errors aren't fatal here
+                pass
+        else:
+            Fetch.write_md5sum(u, ud, d)
 
+        bb.utils.unlockfile(lf)
 
 def checkstatus(d):
     """