I was trying to write a unit test for my modifications to the cache service as part of the implementation of private browsing, and the experience wasn't as smooth as I would have liked. So, I decided to document my experience here.
Firstly, there doesn't seem to be any automated unit tests for the cache service. The only test which uses the cache service directly inside netwerk/test/unit is the test for bug 356133. There are a bunch of stand-alone js and C++ tests residing in netwerk/test, but looking through the makefile, none of them seem to be built or run by default. Oh, and I know that the cache service code is fairly old, tested, and stable, but still, no unit tests at all? We've got to do better than that!
Moving on to writing the unit test anyway, the first step was to check if all three cache devices (memory, disk, and offline) are available. So, I wrote the following code in order to test this:
var cs = get_CacheService();
var visitor = {
QueryInterface: function (iid) {
const interfaces = [Ci.nsICacheVisitor,
Ci.nsISupports];
if (!interfaces.some(function(v) iid.equals(v)))
throw Cr.NS_ERROR_NO_INTERFACE;
return this;
},
visitDevice: function (deviceID, deviceInfo) {
print(deviceID);
return false;
},
visitEntry: function (deviceID, entryInfo) {
do_throw("nsICacheVisitor.visitEntry should not be called " +
"when checking the availability of devices");
}
};
cs.visitEntries(visitor);
What I expected this to output was something like the following:
memory
disk
offline
But I was quite surprised to find out that the visit operation does not find any devices whatsoever! After some time spent reading the source, I figured out that the cache service initializes the devices lazily. In other words, when the cache service is initialized, none of the cache devices get created, and they would each get created on a per-use basis. So, what I did was write some code to add a cache entry for each type of device, and then try to check the availability of the devices. The code I wrote was based on the storeCache function, and looked like this:
function store_in_cache(aKey, aContent, aWhere) {
var storageFlag, streaming = true;
if (aWhere == "disk")
storageFlag = Ci.nsICache.STORE_ON_DISK;
else if (aWhere == "offline")
storageFlag = Ci.nsICache.STORE_OFFLINE;
else if (aWhere == "memory")
storageFlag = Ci.nsICache.STORE_IN_MEMORY;
var cache = get_CacheService();
var session = cache.createSession("PrivateBrowsing", storageFlag, streaming);
var cacheEntry = session.openCacheEntry(aKey, Ci.nsICache.ACCESS_WRITE, true);
var oStream = cacheEntry.openOutputStream(0);
var written = oStream.write(aContent, aContent.length);
if (written != aContent.length) {
do_throw("oStream.write has not written all data!\n" +
" Expected: " + aContent.length + "\n" +
" Actual: " + written + "\n");
}
oStream.close();
cacheEntry.close();
}
store_in_cache("cache-A", "test content", "memory");
store_in_cache("cache-B", "test content", "disk");
store_in_cache("cache-C", "test content", "offline");
So, all was well when testing this on a debug xpcshell, so I decided to run the code on an optimizied non-debug build of xpcshell as well (like I always do), and boom! An NS_ERROR_FAILURE was being thrown on the nsICacheSession.openCacheEntry call. I tested this again and again, but I kept getting the same error in the non-debug build. After spending some time debugging the error, I found out that the culprit was this code. What happened was the cache service tried to get the profile directory in order to find a place to store the disk-based cache files (which will of course fail in xpcshell environment where the profile is not accessible), and in debug builds, it reverted to using the current process directory since the profile directory was not accessible. In non-debug builds, the code simply detected that there's nowhere to store the cache files, so it reported an error which was thrown as an obscure NS_ERROR_FAILURE to the caller. Providing the profile directory manually did the trick:
var dirSvc = Cc["@mozilla.org/file/directory_service;1"].
getService(Ci.nsIProperties);
var provider = {
getFile: function(prop, persistent) {
persistent.value = true;
if (prop == "ProfLD" ||
prop == "ProfD" ||
prop == "cachePDir")
return dirSvc.get("CurProcD", Ci.nsILocalFile);
throw Cr.NS_ERROR_FAILURE;
},
QueryInterface: function(iid) {
if (iid.equals(Ci.nsIDirectoryProvider) ||
iid.equals(Ci.nsISupports)) {
return this;
}
throw Cr.NS_ERROR_NO_INTERFACE;
}
};
dirSvc.QueryInterface(Ci.nsIDirectoryService).registerProvider(provider);
Another oddity I encountered was when trying to read from the cache. A read from the cache should either lead to success, or failure by throwing NS_ERROR_CACHE_KEY_NOT_FOUND from nsICacheSession.openCacheEntry. So, I happily wrote the following code, hoping that all would be well.
function retrieve_from_cache(aKey, aWhere) {
var storageFlag, streaming = true;
if (aWhere == "disk")
storageFlag = Ci.nsICache.STORE_ANYWHERE;
else if (aWhere == "offline")
storageFlag = Ci.nsICache.STORE_OFFLINE;
else if (aWhere == "memory")
storageFlag = Ci.nsICache.STORE_ANYWHERE;
var cache = get_CacheService();
var session = cache.createSession("PrivateBrowsing", storageFlag, streaming);
try {
var cacheEntry = session.openCacheEntry(aKey, Ci.nsICache.ACCESS_READ, true);
} catch (e) {
if (e.result == Cr.NS_ERROR_CACHE_KEY_NOT_FOUND)
return null;
else
do_throw(e);
}
var iStream = make_input_stream_scriptable(cacheEntry.openInputStream(0));
var read = iStream.read(iStream.available());
iStream.close();
cacheEntry.close();
return read;
}
But things were not that easy. The problem was that inside the private browsing mode (where the disk and offline cache devices are turned off and disabled), trying to read an entry from the disk device lead to the NS_ERROR_CACHE_KEY_NOT_FOUND exception, while trying to read data from the offline device lead to NS_ERROR_FAILURE. The fix was easy: just handle this error as another possible result value on the exception.
Finally, the whole test looked like this.
const Cc = Components.classes;
const Ci = Components.interfaces;
const Cr = Components.results;
var _PBSvc = null;
function get_PBSvc() {
if (_PBSvc)
return _PBSvc;
try {
_PBSvc = Cc["@mozilla.org/privatebrowsing;1"].
getService(Ci.nsIPrivateBrowsingService);
if (_PBSvc) {
var observer = {
QueryInterface: function (iid) {
const interfaces = [Ci.nsIObserver,
Ci.nsISupports];
if (!interfaces.some(function(v) iid.equals(v)))
throw Cr.NS_ERROR_NO_INTERFACE;
return this;
},
observe: function (subject, topic, data) {
subject.QueryInterface(Ci.nsISupportsPRUint32);
subject.data = 0;
}
};
var os = Cc["@mozilla.org/observer-service;1"].
getService(Ci.nsIObserverService);
os.addObserver(observer, "private-browsing-enter", false);
}
return _PBSvc;
} catch (e) {}
return null;
}
var _CSvc = null;
function get_CacheService() {
if (_CSvc)
return _CSvc;
return _CSvc = Cc["@mozilla.org/network/cache-service;1"].
getService(Ci.nsICacheService);
}
var cache_prefs = {
"browser.cache.disk.enable" : null,
"browser.cache.offline.enable" : null,
"browser.cache.memory.enable" : null
};
function set_cache_prefs() {
var prefs = Cc["@mozilla.org/preferences-service;1"].
getService(Ci.nsIPrefBranch);
for (var pref in cache_prefs) {
cache_prefs[pref] = prefs.getBoolPref(pref);
prefs.setBoolPref(pref, true);
}
}
function restore_cache_prefs() {
var prefs = Cc["@mozilla.org/preferences-service;1"].
getService(Ci.nsIPrefBranch);
for (var pref in cache_prefs)
prefs.setBoolPref(pref, cache_prefs[pref]);
}
function setup_profile_dir() {
var dirSvc = Cc["@mozilla.org/file/directory_service;1"].
getService(Ci.nsIProperties);
var provider = {
getFile: function(prop, persistent) {
persistent.value = true;
if (prop == "ProfLD" ||
prop == "ProfD" ||
prop == "cachePDir")
return dirSvc.get("CurProcD", Ci.nsILocalFile);
throw Cr.NS_ERROR_FAILURE;
},
QueryInterface: function(iid) {
if (iid.equals(Ci.nsIDirectoryProvider) ||
iid.equals(Ci.nsISupports)) {
return this;
}
throw Cr.NS_ERROR_NO_INTERFACE;
}
};
dirSvc.QueryInterface(Ci.nsIDirectoryService).registerProvider(provider);
}
function check_devices_available(devices) {
var cs = get_CacheService();
var found_devices = [];
var visitor = {
QueryInterface: function (iid) {
const interfaces = [Ci.nsICacheVisitor,
Ci.nsISupports];
if (!interfaces.some(function(v) iid.equals(v)))
throw Cr.NS_ERROR_NO_INTERFACE;
return this;
},
visitDevice: function (deviceID, deviceInfo) {
found_devices.push(deviceID);
return false;
},
visitEntry: function (deviceID, entryInfo) {
do_throw("nsICacheVisitor.visitEntry should not be called " +
"when checking the availability of devices");
}
};
// get the list of active devices
cs.visitEntries(visitor);
// see if any of the required devices was missing
for (var i = 0; i < devices.length; ++ i) {
if (!found_devices.some(function(v) v == devices[i])) {
do_throw("Expected to find the \"" + devices[i] + "\" cache device, " +
"but that device was not available");
}
}
// see if any extra devices have been found
if (found_devices.length > devices.length) {
do_throw("Expected to find these devices: [" + devices.join(", ") +
"], but instead got: [" + found_devices.join(", ") + "]");
}
}
function get_device_entry_count(device) {
var cs = get_CacheService();
var entry_count = -1;
var visitor = {
QueryInterface: function (iid) {
const interfaces = [Ci.nsICacheVisitor,
Ci.nsISupports];
if (!interfaces.some(function(v) iid.equals(v)))
throw Cr.NS_ERROR_NO_INTERFACE;
return this;
},
visitDevice: function (deviceID, deviceInfo) {
if (device == deviceID)
entry_count = deviceInfo.entryCount;
return false;
},
visitEntry: function (deviceID, entryInfo) {
do_throw("nsICacheVisitor.visitEntry should not be called " +
"when checking the availability of devices");
}
};
// get the device entry count
cs.visitEntries(visitor);
return entry_count;
}
function store_in_cache(aKey, aContent, aWhere) {
var storageFlag, streaming = true;
if (aWhere == "disk")
storageFlag = Ci.nsICache.STORE_ON_DISK;
else if (aWhere == "offline")
storageFlag = Ci.nsICache.STORE_OFFLINE;
else if (aWhere == "memory")
storageFlag = Ci.nsICache.STORE_IN_MEMORY;
var cache = get_CacheService();
var session = cache.createSession("PrivateBrowsing", storageFlag, streaming);
var cacheEntry = session.openCacheEntry(aKey, Ci.nsICache.ACCESS_WRITE, true);
var oStream = cacheEntry.openOutputStream(0);
var written = oStream.write(aContent, aContent.length);
if (written != aContent.length) {
do_throw("oStream.write has not written all data!\n" +
" Expected: " + aContent.length + "\n" +
" Actual: " + written + "\n");
}
oStream.close();
cacheEntry.close();
}
function make_input_stream_scriptable(input)
{
var wrapper = Cc["@mozilla.org/scriptableinputstream;1"].
createInstance(Ci.nsIScriptableInputStream);
wrapper.init(input);
return wrapper;
}
function retrieve_from_cache(aKey, aWhere) {
var storageFlag, streaming = true;
if (aWhere == "disk")
storageFlag = Ci.nsICache.STORE_ANYWHERE;
else if (aWhere == "offline")
storageFlag = Ci.nsICache.STORE_OFFLINE;
else if (aWhere == "memory")
storageFlag = Ci.nsICache.STORE_ANYWHERE;
var cache = get_CacheService();
var session = cache.createSession("PrivateBrowsing", storageFlag, streaming);
try {
var cacheEntry = session.openCacheEntry(aKey, Ci.nsICache.ACCESS_READ, true);
} catch (e) {
if (e.result == Cr.NS_ERROR_CACHE_KEY_NOT_FOUND ||
e.result == Cr.NS_ERROR_FAILURE)
// a key not found error is expected here, so we will simply return null
// to let the caller know that no data was retrieved. We also expect
// a generic failure error in case of the offline cache.
return null;
else
do_throw(e); // throw the textual error description
}
var iStream = make_input_stream_scriptable(cacheEntry.openInputStream(0));
var read = iStream.read(iStream.available());
iStream.close();
cacheEntry.close();
return read;
}
function run_test() {
var pb = get_PBSvc();
if (pb) { // Private Browsing might not be available
// Simulate a profile dir for xpcshell
setup_profile_dir();
// Enable all three cache devices
set_cache_prefs();
var cs = get_CacheService();
do_check_neq(cs, null);
// Start off with an empty cache
cs.evictEntries(Ci.nsICache.STORE_ANYWHERE);
// Store cache-A, cache-B and cache-C
store_in_cache("cache-A", "test content", "memory");
store_in_cache("cache-B", "test content", "disk");
store_in_cache("cache-C", "test content", "offline");
// Make sure all three cache devices are available initially
check_devices_available(["memory", "disk", "offline"]);
// Check if cache-A, cache-B and cache-C are avilable
do_check_eq(retrieve_from_cache("cache-A", "memory"), "test content");
do_check_eq(retrieve_from_cache("cache-B", "disk"), "test content");
do_check_eq(retrieve_from_cache("cache-C", "offline"), "test content");
// Enter private browsing mode
pb.privateBrowsing = true;
// Make sure none of cache-A, cache-B and cache-C are available
do_check_eq(retrieve_from_cache("cache-A", "memory"), null);
do_check_eq(retrieve_from_cache("cache-B", "disk"), null);
do_check_eq(retrieve_from_cache("cache-C", "offline"), null);
// Make sure only the memory device is available
check_devices_available(["memory"]);
// Make sure the memory device is empty
do_check_eq(get_device_entry_count("memory"), 0);
// Exit private browsing mode
pb.privateBrowsing = false;
// Make sure all three cache devices are available after leaving the private mode
check_devices_available(["memory", "disk", "offline"]);
// Check if cache-A is gone, and cache-B and cache-C are still avilable
do_check_eq(retrieve_from_cache("cache-A", "memory"), null);
do_check_eq(retrieve_from_cache("cache-B", "disk"), "test content");
do_check_eq(retrieve_from_cache("cache-C", "offline"), "test content");
// Restore cache device prefs to their default values
restore_cache_prefs();
}
}
The cache service really needs some unit test, and documentation love. Nice things to see documented would be the lazy initialization of devices, and the fact that the cache service picks up the process directory in debug mode as the cache directory. I guess all of these can be covered in some sort of tutorial to the cache service APIs. Anyway, until such a turotial gets written, I hope this post would make the life of people who decide to use the cache service a bit easier.