History log of /haiku/src/system/kernel/cache/file_cache.cpp
Revision Date Author Comments
# a9be0efb 31-Aug-2019 Augustin Cavalier <waddlesplash@gmail.com>

kernel/fs: Add support for setting custom VMCaches in vnodes.

This adds one (private) VFS function, and checks in all usages of
the vnode->cache as a VMVnodeCache that it really is one. (Generic
usages, for the moment just the ReleaseRef() calls in vnode
destruction, are intentionally not touched.)

This will be used by ramfs to set the cache from its own,
so that map_file() calls on a ramfs can work.


# 728b515e 29-Apr-2019 Augustin Cavalier <waddlesplash@gmail.com>

kernel: Bounds-check the parameters passed to file_cache_read().

cache_io() did some bounds checking, but for uncached reads it was
entirely on the filesystem. Now we are more consistent: do all the
bounds checking for reads in the file cache, and do all bounds
checking for writes in the filesystems.

This makes sense as nearly all file systems were doing the exact
same logic for read() but of course all have different logic
for write(), due to block allocation, etc.

This potentially fixes #14993.

Change-Id: Iaf3e549001344cf375c7b8de549fc169d77bdbb2
Reviewed-on: https://review.haiku-os.org/c/1420
Reviewed-by: waddlesplash <waddlesplash@gmail.com>


# 8844a67e 10-Dec-2018 Augustin Cavalier <waddlesplash@gmail.com>

More trivial syntax and logic cleanup.

Spotted by Clang. No functional change intended.


# 9c5c5990 15-Sep-2014 Paweł Dziepak <pdziepak@quarnos.org>

kernel: pagecache: provided buffers are not always in user memory

Source or destination buffers passed to pagecache functions may belong
to kernel memory (e.g. when the caller is packagefs). Because of that
we should tell vm_memcpy_{from, to}_physical() truth, not assume that all
buffers are in user memory. That's important because user memory page fault
handlers cannot be nested and these functions may be used while handling
a page fault.

With high probability fixes #11246.

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>


# 0e88a887 13-Jun-2012 Alex Smith <alex@alex-smith.me.uk>

First round of 64-bit safety fixes in the kernel.

* Most of this is incorrect printf format strings. Changed all strings
causing errors to use the B_PRI* format string definitions, which
means the strings should be correct across all platforms.
* Some other fixes for errors, casts required, etc.


# f8154d17 02-Nov-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

mmlr (distracted) + bonefish:
* Turn VMCache::consumers C list into a DoublyLinkedList.
* Use object caches for the different VMCache types and the VMCacheRefs.
The purpose is to reduce slab area fragmentation.
* Requires the introduction of a pure virtual VMCache::DeleteObject()
method, implemented in the derived classes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43133 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 222b8cf7 18-May-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

cache_io(): Since satisfy_cache_io() (respectively the function it calls)
unlocks the cache, we have to look up the page again, since someone else could
have removed it in the meantime. Possibly fixes #7514.

+r1alpha3


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@41564 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1d578e15 02-Jun-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

Fixed more address types related issues. Mostly printf() or comparison
warnings, but also some oversights from earlier changes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37000 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 435c43f5 02-Jun-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* Introduced type generic_io_vec, which is similar to iovec, but uses types
that are wide enough for both virtual and physical addresses.
* DMABuffer, IORequest, IOScheduler,... and code using them: Use
generic_io_vec and generic_{addr,size}_t where necessary.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36997 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4fe399e4 28-Apr-2010 Axel Dörfler <axeld@pinc-software.de>

* Use vm_page_write_modified_page_range() instead of
vm_page_schedule_write_page_range() to prevent more memory pressure.
* While it seems to help a bit, it doesn't solve the problem of bug #5777.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36519 a95241bf-73f2-0310-859d-f6bbb57e9c96


# bd7645a1 20-Feb-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* Made vm_page::state private and added accessor methods.
* Added kernel tracing for page state transitions.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35538 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 211c63df 19-Feb-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

reserve_pages(): When cannibalizing pages from the own cache it didn't check
whether the pages where mapped. Was introduced already in r22731.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35532 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9824ded0 20-Feb-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

cache_io(): Also requeue the last touched page to keep things roughly LRU
sorted.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35530 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1f611c01 16-Feb-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

The file cache code was completely ignoring the vm_page::modified flag. In
particular it wouldn't set the flag when writing something to a page, but
only move it to the modified queue. Since mapping the page would move it to
another queue, the information that the page was modified would be lost and
it would never be written to disk. Was well reproducible with a Haiku image
build and limited amount of memory.
Fixes the hopefully last remaining cause for #5374.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35500 a95241bf-73f2-0310-859d-f6bbb57e9c96


# cee04e80 08-Feb-2010 Artur Wyszynski <aljen-mlists@o2.pl>

Fixed various errors/warnings reported by cppcheck:
* memory leaks
* resource leaks
* added const's to getters
* removed a few reundant conditions


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35433 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 40bb9481 03-Feb-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* Removed useless return parameter from vm_remove_all_page_mappings().
* Added vm_clear_page_mapping_accessed_flags() and
vm_remove_all_page_mappings_if_unaccessed(), which combine the functionality
of vm_test_map_activation(), vm_clear_map_flags(), and
vm_remove_all_page_mappings(), thus saving lots of calls to translation map
methods. The backend is the new method
VMTranslationMap::ClearAccessedAndModified().
* Started to make use of the cached page queue and changed the meaning of the
other non-free queues slightly:
- Active queue: Contains mapped pages that have been used recently.
- Inactive queue: Contains mapped pages that have not been used recently. Also
contains unmapped temporary pages.
- Modified queue: Contains unmapped modified pages.
- Cached queue: Contains unmapped unmodified pages (LRU sorted).
Unless we're actually low on memory and actively do paging, modified and
cached queues only contain non-temporary pages. Cached pages are considered
quasi free. They still belong to a cache, but since they are unmodified and
unmapped, they can be freed immediately. And this is what
vm_page_[try_]reserve_pages() do now when there are no more actually free
pages at hand. Essentially this means that pages storing cached file data,
unless mmap()ped, no longer are considered used and don't contribute to page
pressure. Paging will not happen as long there are enough free + cached pages
available.
* Reimplemented the page daemon. It no longer scans all pages, but instead works
the page queues. As long as the free pages situation is harmless, it only
iterates through the active queue and deactivates pages that have not been
used recently. When paging occurs it additionally scans the inactive queue and
frees pages that have not been used recently.
* Changed the page reservation/allocation interface:
vm_page_[try_]reserve_pages(), vm_page_unreserve_pages(), and
vm_page_allocate_page() now take a vm_page_reservation structure pointer.
The reservation functions initialize the structure -- currently consisting
only of a count member for the number of still reserved pages.
vm_page_allocate_page() decrements the count and vm_page_unreserve_pages()
unreserves the remaining pages (if any). Advantages are that reservation/
unreservation mismatches cannot occur anymore, that vm_page_allocate_page()
can verify that the caller has indeed a reserved page left, and that there's
no unnecessary pressure on the free page pool anymore. The only disadvantage
is that the vm_page_reservation object needs to be passed around a bit.
* Reworked the page reservation implementation:
- Got rid of sSystemReservedPages and sPageDeficit. Instead
sUnreservedFreePages now actually contains the number of free pages that
have not yet been reserved (it cannot become negative anymore) and the new
sUnsatisfiedPageReservations contains the number of pages that are still
needed for reservation.
- Threads waiting for reservations do now add themselves to a waiter queue,
which is ordered by descending priority (VM priority and thread priority).
High priority waiters are served first when pages become available.
Fixes #5328.
* cache_prefetch_vnode(): Would reserve one less page than allocated later, if
the size wasn't page aligned.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35393 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e65c4002 29-Jan-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* Replaced the vm_page_allocate_page*() "pageState" parameter by a more
general "flags" parameter. It encodes the target state of the page -- so
that the page isn't unnecessarily put in the wrong page queue first -- a
flag whether the page should be cleared, and one to indicate whether the
page should be marked busy.
* Added page state PAGE_STATE_CACHED. Not used yet.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35333 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 72382fa6 29-Jan-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* Removed the page state PAGE_STATE_BUSY and instead introduced a vm_page::busy
flag. The obvious advantage is that one can still see what state a page is in
and even move it between states while being marked busy.
* Removed the vm_page::is_dummy flag. Instead we mark marker pages busy, which
in all cases has the same effect. Introduced a vm_page_is_dummy() that can
still check whether a given page is a dummy page.
* vm_page_unreserve_pages(): Before adding to the system reserve make sure
sUnreservedFreePages is non-negative. Otherwise we'd make nonexisting pages
available for allocation. steal_pages() still has the same problem and it
can't be solved that easily.
* map_page(): No longer changes the page state/mark the page unbusy. That's the
caller's responsibility.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35331 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 64a9bd15 26-Jan-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

Optimized writing of zeroes to files. Instead of allocating a small buffer
on the fly, clearing and writing it each time, we now use an iovec with 32
identical entries pointing to a clear page that we prepare once at
initialization. This speeds up clear_image in low memory situations
dramatically.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35304 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 7fa3e4e6 26-Jan-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

cache_io(): Reevaluate the memory situation periodically. Deciding at the
beginning whether to pass the cache by really doesn't help when
reading/writing a huge amount of data, since a low memory situation is likely
to occur at some point during the operation. This should fix the main issue
of #3768.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35299 a95241bf-73f2-0310-859d-f6bbb57e9c96


# cff6e9e4 26-Jan-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* The system now holds back a small reserve of committable memory and pages. The
memory and page reservation functions have a new "priority" parameter that
indicates how deep the function may tap into that reserve. The currently
existing priority levels are "user", "system", and "VIP". The idea is that
user programs should never be able to cause a state that gets the kernel into
trouble due to heavy battling for memory. The "VIP" level (not really used
yet) is intended for allocations that are required to free memory eventually
(in the page writer). More levels are thinkable in the future, like "user real
time" or "user system server".
* Added "priority" parameters to several VMCache methods.
* Replaced the map_backing_store() "unmapAddressRange" parameter by a "flags"
parameter.
* Added area creation flag CREATE_AREA_PRIORITY_VIP and slab allocator flag
CACHE_PRIORITY_VIP indicating the importance of the request.
* Changed most code to pass the right priorities/flags.

These changes already significantly improve the behavior in low memory
situations. I've tested a bit with 64 MB (virtual) RAM and, while not
particularly fast and responsive, the system remains at least usable under high
memory pressure.
As a side effect the slab allocator can now be used as general memory allocator.
Not done by default yet, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35295 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3cd20943 06-Jan-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* Added new debug feature (DEBUG_PAGE_ACCESS) to detect invalid concurrent
access to a vm_page. It is basically an atomically accessed thread ID field
in the vm_page structure, which is explicitly set by macros marking the
critical sections. As a first positive effect I had to review quite a bit of
code and found several issues.
* Added several TODOs and comments. Some harmless ones, but also a few
troublesome ones in vm.cpp regarding page unmapping.
* file_cache: PrecacheIO::Prepare()/read_into_cache: Removed superfluous
vm_page_allocate_page() return value checks. It cannot fail anymore.
* Removed the heavily contended "pages" lock. We use different policies now:
- sModifiedTemporaryPages is accessed atomically.
- sPageDeficitLock and sFreePageCondition are protected by a new mutex.
- The page queues have individual locks (mutexes).
- Renamed set_page_state_nolock() to set_page_state(). Unless the caller says
otherwise, it does now lock the affected pages queues itself. Also changed
the return value to void -- we panic() anyway.
* set_page_state(): Add free/clear pages to the beginning of their respective
queues as this is more cache-friendly.
* Pages with the states PAGE_STATE_WIRED or PAGE_STATE_UNUSED are no longer
in any queue. They were in the "active" queue, but there's no good reason
to have them there. In case we decide to let the page daemon work the queues
(like FreeBSD) they would just be in the way.
* Pulled the common part of vm_page_allocate_page_run[_no_base]() into a helper
function. Also fixed a bug I introduced previously: The functions must not
vm_page_unreserve_pages() on success, since they remove the pages from the
free/clear queue without decrementing sUnreservedFreePages.
* vm_page_set_state(): Changed return type to void. The function cannot really
fail and no-one was checking it anyway.
* vm_page_free(), vm_page_set_state(): Added assertion: The page must not be
free/clear before. This is implied by the policy that no-one is allowed to
access free/clear pages without holding the respective queue's lock, which is
not the case at this point. This found the bug fixed in r34912.
* vm_page_requeue(): Added general assertions. panic() when requeuing of
free/clear pages is requested. Same reason as above.
* vm_clone_area(), B_FULL_LOCK case: Don't map busy pages. The implementation is
still not correct, though.

My usual -j8 Haiku build test runs another 10% faster, now. The total kernel
time drops about 18%. As hoped the new locks have only a fraction of the old
"pages" lock contention. Other locks lead the "most wanted list" now.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34933 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1021fd28 01-Jan-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* agp_gart(): Use vm_page_[un]reserve_pages().
* Removed unused vm_page_allocate_pages().
* Removed now unused (always true) "reserved" parameter from
vm_page_allocate_page().
* Removed unused (always false) "stealActive" parameter from steal_page().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34836 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 522c2f19 07-Dec-2009 Ingo Weinhold <ingo_weinhold@gmx.de>

* Added a simple mechanism to wait for events to VMCache. WaitForPageEvents()
waits for certain events on a given page, NotifyPageEvents() wakes up
waiting threads respectively.
* Used the new feature instead of condition variables for waiting on busy
pages. We save publishing and unpublishing of a condition variable whenever
a page is marked busy. There's only something to do, if there's at least
one thread waiting in the list of the respective cache. The general
assumption is that this is only rarely the case and even if it happens,
there should be only very few threads.
* Added an apparently missing notification in cache_io(). At least I didn't
see the reason for it not being there.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34537 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e50cf876 02-Dec-2009 Ingo Weinhold <ingo_weinhold@gmx.de>

* Moved the VM headers into subdirectory vm/.
* Renamed vm_cache.h/vm_address_space.h to VMCache.h/VMAddressSpace.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34449 a95241bf-73f2-0310-859d-f6bbb57e9c96


# a477e3cf 01-Dec-2009 Ingo Weinhold <ingo_weinhold@gmx.de>

Finished renaming of vm_cache to VMCache.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34424 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1af7d115 10-Oct-2009 Michael Lotz <mmlr@mlotz.ch>

* Rework page writing to combine page writes where possible. For now the pages
are required to be physically contiguos, which should be reworked to put them
into seperate iovecs. Still this manages to combine a great deal of page
writes into larger bursts already. Reduces the amount of IO requests being
scheduled (and greatly benefits media where page wise writes are slow when
they are accessed through a non-IOScheduler path, i.e. USB mass storage until
that is properly implemented).
* Abstracted per page page writing tasks into a PageWriteWrapper class.
* Abstracted per transfer page writing tasks into PageWriteTransfer class which
formerly was the PageWriterCallback.
* Use both classes from the PageWriterRun and from
vm_page_write_modified_page_range to remove code duplication.
* Adjusted synchronous VMAnonymousCache::Write() to cope correctly with larger
iovecs and more than one iovec. It assumed that there was exactly one page per
vector previously.
* Introduced MaxPagesPerWrite() and MaxPagesPerAsyncWrite() to VMCache to allow
a cache to specify restricitions. VMAnonymousCache does restrict the max pages
to 1 for WriteAsync right now as I didn't feel like reworking that one to cope
with non single page writes just yet.
* Pulled out PageWriteTransfer methods for better readability.
* Some typo fixes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33507 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 61b6f38c 31-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* Cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@32021 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c32499b4 30-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* Since the file_map reports its chunk with a file system specific alignment, we
cannot assume that the file is that long - we need to check for the actual
file size specifically, and then clear the extraneous bytes based on that one.
* This fixes seeing old file left-overs in the space beyond the file size in
mmapped file. Thanks to Oliver who provided me with a nice test program to
reproduce this problem (this should also fix gcc's fix_includes app (although
I wonder what it does if a file actually ends on a page boundary).
* Also fixed a bug in precaching that would cause the last page of the cache
to be discarded, causing it to be re-read later on.
* Moved partial page clearing into its own function
read_pages_and_clear_partial(), even though it's currently only used once.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@32001 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 6102f078 24-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* file_cache_set_size() now clears the last page of its cache if needed. This
fixes #4106 for good.
* Also removed a bit of dead code in that function.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31732 a95241bf-73f2-0310-859d-f6bbb57e9c96


# db8fb4fd 21-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* The file cache must clear partial pages as well, since they can be mapped
directly via mmap().
* This fixes bug #4106.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31669 a95241bf-73f2-0310-859d-f6bbb57e9c96


# a5dbd78b 05-May-2009 Axel Dörfler <axeld@pinc-software.de>

* Since PrecacheIO is used as asynchronous callback object, we must not access
it anymore after having called vfs_asynchronous_read_pages().
* Now, Prepare() does all the preparation work, and ReadAsync() does the actual
work - this must be called without having the cache locked. This also fixes
another bug where the callback would be deleted twice in case the I/O request
failed.
* This fixes bug #3847.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30632 a95241bf-73f2-0310-859d-f6bbb57e9c96


# d5ad7629 30-Apr-2009 Axel Dörfler <axeld@pinc-software.de>

Fixed several problems of the prefetching code:
* Did claim to have reserved pages when calling vm_page_allocate_page(), but
didn't have any (copy&paste bug). We cannot use it without reserved pages,
as we need to call vm_page_allocate_page() with a cache locked.
* No longer use low_resource_state() to determine whether to precache or not,
but use the new vm_page_num_used_pages() instead.
* Also don't (try to) precache when the cache already has more than 2/3 of its
pages to safe some unnecessary work.
* The size to precache was limited to the file size incorrectly.
* When precaching failed, the cache reference was not released.
* The precaching started one page too late, causing bug #3835.
* Reenabled precaching.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30515 a95241bf-73f2-0310-859d-f6bbb57e9c96


# aff0fbbe 28-Apr-2009 Stephan Aßmus <superstippi@gmx.de>

axeld:
* vfs_read_pages_async() must not be called with locked cache,
since it cannot be guaranteed that the operation will be
performed asynchronously. (The ISO9660 FS for example does not
implement the new IO hooks... yet.)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30476 a95241bf-73f2-0310-859d-f6bbb57e9c96


# eb2bd0e8 27-Apr-2009 Stephan Aßmus <superstippi@gmx.de>

axeld:
* Implemented a way to do asynchronous pre-fetching when mapping files.
* There are slight code duplications in some places that could benefit
from cleaning up, but nothing too bad.
* Implementing smarter ways to trigger prefetching and more analysis of
the situations in the kernel would be nice. Currently up to 10 MB
of every mapped file are pre-fetched without further analysis.
* The speed improvement is nice for certain operations. On our test
system (real hardware), Firefox took 9 seconds from being launched
to display a window. Now it takes 5 seconds. Both measurements
right after booting. The same system took 35 seconds from launching
Haiku in the GRUB menu to displaying the Tracker desktop background
image. Now it takes 27 seconds.
* We didn't have the chance to check out the effects of this on the
CD boot, but potentially, they could speed it up a lot.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30465 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 8a26f35a 20-Apr-2009 Axel Dörfler <axeld@pinc-software.de>

* Sequential write accesses were never detected, due to an incorrect check.
* The previous code would have scheduled a single page to be written out (if it
would have ever been triggered), now we schedule the complete previous write
access. This greatly speeds up a "dd if=/dev/zero of=test ..." beyond the
size of available memory.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30276 a95241bf-73f2-0310-859d-f6bbb57e9c96


# aa4ba93e 08-Mar-2009 Ingo Weinhold <ingo_weinhold@gmx.de>

* Renamed src/system/kernel/device_manager/io_requests.{h,cpp} to
IORequest.{h,cpp}.
* Introduced public <io_requests.h> header. Currently it only declares the
single function BFS uses.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29446 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 47c40a10 19-Oct-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* Prefixed memset_physical() and memcpy_to_physical() with "vm_",
added vm_memcpy_from_physical() and vm_memcpy_physical_page(), and
added respective functions to the vm_translation_map operations. The
architecture specific implementation can now decide how to implement
them most efficiently. Added generic implementations that can be used,
though.
* Changed vm_{get,put}_physical_page(). The former no longer accepts
flags (the only flag PHYSICAL_PAGE_DONT_WAIT wasn't needed anymore).
Instead it returns an implementation-specific handle that has to be
passed to the latter. Added vm_{get,put}_physical_page_current_cpu()
and *_debug() variants, that work only for the current CPU,
respectively when in the kernel debugger. Also adjusted the
vm_translation_map operations accordingly.
* Made consequent use of the physical memory operations in the source
tree.
* Also adjusted the m68k and ppc implementations with respect to the
vm_translation_map operation changes, but they are probably broken,
nevertheless.
* For x86 the generic physical page mapper isn't used anymore. It is
suboptimal in any case. For systems with small memory it is too much
overhead, since one can just map the complete physical memory (that's
not done yet, though). For systems with large memory it counteracts
the VM strategy to reuse the least recently used pages. Since those
pages will most likely not be mapped by the page mapper anymore, it
will keep remapping chunks. This was also the reason why building
Haiku in Haiku was significantly faster with only 256 MB RAM (since
that much could be kept mapped all the time).
Now we're using a different strategy: We have small pools of virtual
page slots per CPU that are used for the physical page operations
(memset_physical(), memcpy_*_physical()) with CPU-pinned thread.
Furthermore we have four slots per translation map, which are used to
map page tables.

These changes speed up the Haiku image build in Haiku significantly. On
my Core2 Duo 2.2 GHz 2 GB machine about 40% to 20 min 40 s (KDEBUG
disabled, block cache debug disabled). Still more than factor 3 slower
than FreeBSD and Linux, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28244 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 7b8683b2 17-Oct-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Updated write_to_cache() to use physical pages for I/O and the new
{memset,memcpy_to}_physical() functions.
Mapping lots of physical pages at once as done before was an actual
problem on systems with enough RAM, as the physical page mapper can map
only 64 chunks at a time. So multiple threads could play dining
philosophers, each getting only one of two chopsticks, waiting for
another one to be freed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28220 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1b6eff28 11-Oct-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* Replaced the vm_get_physical_page() "flags"
PHYSICAL_PAGE_{NO,CAN}_WAIT into an actual flag
PHYSICAL_PAGE_DONT_WAIT.
* Pass the flags through to the chunk mapper callback.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27979 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3c1a3047 04-Aug-2008 Axel Dörfler <axeld@pinc-software.de>

* Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 7491000f 03-Aug-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Added functions file_cache_{disable,enable}(). They allow to disable
actual caching in the file cache, i.e. all reads and writes go directly
to the underlying device. The implementation is not quite complete,
since the VM can still add pages to the cache when the file is mmap()ed,
which can lead to inconsistencies.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26779 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9a0fc9a8 31-Jul-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

In cache_io(): Don't keep the cache locked while doing a
user_{memcpy,memset}(), since that can cause a page fault, which needs
pages and might try to steal some from our cache.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26700 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 7f12cc54 30-Jul-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* It is now supported that I/O operations and requests are only handled
partially (e.g. due to hitting the end of file). The respective
classes have grown new methods and attributes to deal with that. The
"finished" callbacks have got additional parameters to indicate
whether the transfer was only partial and how much has been
transferred. Other callbacks and functions have a size_t* in/out
parameter instead of a simple size_t, now.
* vfs_{read,write}_pages() do now use the I/O request framework instead
of the underlying FS's {read,write}_pages() hooks (those should be
unused now). Furthermore they've got an additional "flags" parameter,
which is passed to IORequest::Init(), i.e. it allows to specify that
the given vecs refer to physical addresses.
* The file cache's read_into_cache() reads directly into physical
pages, now.
* Fixed bug in DoIO::IO(): The offset was not adjusted, so that all
pages were incorrectly transferred from/to the same location.
* Fixed broken subrequest scheduling loop head in
do_iterative_fd_io_iterate().
* Adjusted the test driver and implemented its io() hook. Using this
driver I/O requests are passed all the way from the VFS/VM to the
driver and through the I/O scheduler. It even seems to work. :-)
* Added missing const to the iovec* parameter of the IORequest::Init()
methods.
* Disabled some debug output by default. Added new optional debug
output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26692 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e6bd90c5 23-Jul-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* bfs_fsync() was the only place which could cause the
fs_vnode_ops::write_pages() to be called with fsReenter = true. Since
this is no longer the case, the argument has become superfluous. For
read_pages() it always was. Removed the argument from the functions
and all functions that propagated it.
* Some whitespace at the end of lines was removed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26579 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 190712ce 23-Jul-2008 Axel Dörfler <axeld@pinc-software.de>

* file_cache_sync() no longer needs the file system's lock.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26577 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5c99d639 22-Jul-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Merged branch haiku/branches/developer/bonefish/vm into trunk. This
introduces the following relevant changes:
* VMCache:
- Renamed vm_cache to VMCache, merged it with vm_store and made it a
C++ class with virtual methods (replacing the store operations).
Turned the different store implementations into subclasses.
- Introduced MergeStore() callback, changed semantics of Commit().
- Changed locking and referencing semantics. A reference can only be
acquired/released with the cache locked. An unreferenced cache is
deleted and a mergeable cache merged when it is unlocked. This
removes the "busy" state of a cache and simplifies the page fault
code.
* Added VMAnonymousCache, which will implement swap support (work by
Zhao Shuai). It is not integrated and used yet, though.
* Enabled the mutex/recursive lock holder asserts.
* Fixed DoublyLinkedList::Swap().
* Generalized the low memory handler to a low resource handler. And made
semaphores and reserved memory handled resources. Made
vm_try_resource_memory() optionally wait (with timeout), and used that
feature to reserve memory for areas.
...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26572 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 13ea2be5 16-Jul-2008 Axel Dörfler <axeld@pinc-software.de>

* Since cache_io() can now work with NULL buffers, I resurrected prefetching
using it.
* IOW cache_prefetch_vnode() should work again now.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26468 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9e1ea0e7 28-Jun-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Don't leak the buffer allocated at the beginning of the function.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26161 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e1b630c5 28-Jun-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* Replaced the global cache pages hash table by an IteratableSplayTree
per cache.
* Changed the strategy vm_cache_acquire_page_cache_ref() uses to ensure
that the cache isn't deleted while trying to get a reference. Instead
of the global cache pages hash table lock, it holds the global cache
list lock now. We acquire + release this lock in delete_cache() after
removing all pages and just before deleting the object.
* Some small optimizations using the property that the cache's pages are
ordered, now (vm_cache_resize(), vm_page_write_modified_page_range(),
vm_page_schedule_write_page_range()).
* Replaced some code counting a cache's pages by simply using
vm_cache::page_count.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26160 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 0c615a01 01-May-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* Removed old mutex implementation and renamed cutex to mutex.
* Trivial adjustments of code using mutexes. Mostly removing the
mutex_init() return value check.
* Added mutex_lock_threads_locked(), which is called with the threads
spinlock being held. The spinlock is released while waiting, of
course. This function is useful in cases where the existence of the
mutex object is ensured by holding the threads spinlock.
* Changed the two instances in the VFS code where an IO context of
another team needs to be locked to use mutex_lock_threads_locked().
Before it required a semaphore-based mutex implementation.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25283 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 184de764 30-Apr-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Replaced the vm_cache mutex by a cutex. This should save quite a few
semaphores.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25277 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 6cef245e 22-Apr-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* Detemplatized ConditionVariable{Entry}. Merged them with their
respective Private* base class.
* Changed sigwait() and sigsuspend() to use thread_block() instead of a
condition variable.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25100 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c12ed856 25-Mar-2008 Axel Dörfler <axeld@pinc-software.de>

Fixed warning.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24581 a95241bf-73f2-0310-859d-f6bbb57e9c96


# a121b8c8 24-Mar-2008 Axel Dörfler <axeld@pinc-software.de>

* Added a new flag for write stat: B_STAT_SIZE_INSECURE that allows a
file system to not fill newly created space with zeros.
BFile::SetSize() now uses this, while [f]truncate() does not. This
is only a temporary work-around until BFS supports sparse files.
* Apps that want to reserve space to fill up later should use
BFile::SetSize() for now, as this will be a lot faster than
[f]truncate().
* cache_io() and the functions below now use a special mode when you
pass in a NULL buffer: for read access, the cache is only populated
(useful for prefetching), for write access, the file is filled with
zeros.
* Implemented BFS's Inode::FillGapWithZeros() using this method now.
* Removed extraneous white space.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24555 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9ff70e74 26-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

read_from_file() and write_to_file() did not take the pageOffset into account.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22998 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3d268eda 10-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

* Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9edc2b52 08-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

The reserved pages computation used the same broken logic as read_into_cache()
and write_to_cache() before, IOW the cache could reserve too few pages.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22859 a95241bf-73f2-0310-859d-f6bbb57e9c96


# cfe386c2 08-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

* read_into_cache() and write_to_cache() both did not compute the number of
needed pages correctly, and would also not read/write enough in case the
offset didn't start at 0 resulting in undetected short reads/writes. It's
amazing how many bugs can be hidden in a few lines of code.
* Fixed a bug that might have been the cause for bug #1601: when the last part
of the write did not end on a page boundary, the last page had to be read
first, but that was done from the wrong offset. Also, if only parts of that
page could be read (because the file size didn't span over the whole page)
the remaining parts needed to be cleared.
* The cache_funcs were always called with the same value for numBytes and
bufferSize so I've eliminated the former.
* Large reads now also bypass the cache in case of low memory, large writes now
also only bypass the cache in that case, following Ingo's suggestion.
* Fixed compilation with debugging turned on.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22858 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 09149281 08-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

The file cache now completely bypasses the cache for writes equal or larger
than 64KB. Reads should probably get a similar logic, at least if memory is
tight.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22857 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c6573329 26-Oct-2007 Axel Dörfler <axeld@pinc-software.de>

* The file cache now remembers the last 3 accesses to be able to detect
sequential access.
* This is now used to let the cache free some pages (or schedule them to be
written back) before vm_reserve_pages() is called, but only if that cache
is not mapped, and there is a memory shortage.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22731 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 0d871d3c 08-Oct-2007 Axel Dörfler <axeld@pinc-software.de>

* satisfy_cache_io() was using the wrong value to compute the number of
pages to reserve, which could result in to few being reserved in certain
situations.
* Use MutexLocker where appropriate.
* Reordered includes following the new rules.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22494 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 0710d59c 08-Oct-2007 Axel Dörfler <axeld@pinc-software.de>

* Added a "canWait" argument to vfs_get_vnode() - you can use this to ignore
busy vnodes.
* dir_create_entry_ref() used get_vnode() incorrectly (and could therefore
potentially prevent a file system from doing proper locking when called
from the kernel).
* The vnode_store now uses this for its acquire_unreferenced_ref()
implementation (and therefore for the page writer).
* read_into_cache() and write_to_cache() were still marked inline.
* The system will now wait 10 secs for a busy vnode before returning an error.
* It will also no longer panic in that case.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22485 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 20b232e9 07-Oct-2007 Axel Dörfler <axeld@pinc-software.de>

Actually forgot this nail: the file cache will now reserve the pages it will
allocate with the vm_cache locked - this is necessary to be able to steal pages
from itself (large files...).
The system doesn't actually lock up anymore, but it still renders itself unusable;
obviously the page thief does not work correctly, yet. The rest of the experience
is created by our current scheduler (the page thief runs and runs, but it doesn't
free any pages anymore).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22466 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 0633dcc2 06-Oct-2007 Axel Dörfler <axeld@pinc-software.de>

* Fixed a serious oversight that could result in trying to insert a page
twice into the same cache: cache_io() called read_into_cache() (or
write_to_cache()), and that broke down the request into smaller parts.
It then called read_chunk_into_cache() (or write_chunk_to_cache() resp.)
to actually allocate pages and fulfill the request.
However, it needed to unlock the cache for each chunk, and in the mean
time someone else could insert pages into the remaining chunks.
* Now, cache_io() already takes care of chunking the data which makes this
approach safe, and also simplified the code a bit - read_into_cache()/
write_to_cache() are gone now. I've renamed read_chunk_into_cache() to
read_into_cache() (same for the write function).
* Also got rid of that goto in that function while I was on it.
* Disabled cache_prefetch_vnode() for now (it's similar to cache_io(), but
since it's currently not used [since no cache module is installed yet],
I didn't want to go through updating it now, too).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22465 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 80f54692 06-Oct-2007 Axel Dörfler <axeld@pinc-software.de>

* struct vnode is an opaque type now, removed void* where it was used incorrectly.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22461 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3e9513aa 03-Oct-2007 Axel Dörfler <axeld@pinc-software.de>

* fs_{write|read}_pages() now has an additional argument "mayBlock".
* the page writer don't allow to block, while all other writers do. This fixes
bug #1509. The reason the page writer needs this is because it marks several
pages from different caches as busy.
* Fixed a warning about ASSERT being defined already in BFS, since
util/DoublyLinkedList.h now includes debug.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22434 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 44e2fd09 26-Sep-2007 Axel Dörfler <axeld@pinc-software.de>

Actually, it's not a good idea to increase the usage_count, since every access
is tracked while the page scanner runs much slower. Now, we just set it to a
fixed positive, so that they can easily age in case they are actually unused.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22320 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 8e0f884c 26-Sep-2007 Axel Dörfler <axeld@pinc-software.de>

* Since the page scanner and thief can work more effectively when no vm_caches
are locked, there is now a vm_page_reserve_pages() call to ensure upfront that
there is a page for me when I need it, and may have locked some caches.
* The vm_soft_fault() routine now makes use of that feature.
* vm_page_allocate_page() now resets the vm_page::usage_count, so that the file
cache does not need to do this in read_chunk_into_cache() and
write_chunk_to_cache().
* In cache_io() however, it need to update the usage_count - and it does that
now. Since non-mapped caches don't have mappings, the page scanner will punish
the cache pages stronger than other pages which is accidently just what we
want.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22319 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 6d4aea47 25-Sep-2007 Axel Dörfler <axeld@pinc-software.de>

bonefish+axeld:
* Removed the vm_cache/vm_store ref_count duality that besides being a bit ugly
also created the page dameon cache retrieval problem: now, only areas (and
cache consumers) retrieve a reference to the store (and therefore, the vnode).
The page daemon doesn't need to care about this at all anymore, and the pseudo
references of the vm_cache could be removed again.
* Rearranged deletion of vnodes such that its ID can be reused directly after
fs_remove_vnode() has been called.
* vm_page_allocate_page() no longer panics when it runs out of pages, but just
waits for new pages to become available using the new sFreeCondition condition
variable - to make sure this happens in an acceptable time frame, it'll
trigger a run of the low memory handlers.
* Implemented a page_thief() that steals inactive pages from caches and puts
them into the free queue. It runs as a low memory handler.
* The file cache now sets the usage count on the pages it inserts into the
cache (needs some rework though, cache_io() doesn't do it yet).
* Instead of panicking, the kernel will currently dead lock in low memory
situations, since BFS does a bit too much in bfs_release_vnode().
* Some minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22315 a95241bf-73f2-0310-859d-f6bbb57e9c96


# b50494aa 02-Sep-2007 Axel Dörfler <axeld@pinc-software.de>

* file_cache_set_size() now calls file_cache_invalidate_file_map() with the
correct values (resolving a TODO); however, the latter doesn't do anything
with those yet.
* Cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22154 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 13542314 10-Aug-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

Removed special dummy page handling. It became obsolete with r21816,
since fault_find_page() does no longer insert a dummy page into a cache
that has a store from which it can read the page.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21887 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 279c6b76 09-Aug-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

* Use condition variables when waiting for busy pages or busy caches.
* Removed a few instances where the page state was set busy directly after
allocating it. This is a no-op, since a page is always busy after
allocation.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21875 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 7c3a45ec 03-Aug-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

Although probably not used ATM, cache_prefetch_vnode() should respect
busy pages.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21815 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 58f6e8e5 17-Jul-2007 Axel Dörfler <axeld@pinc-software.de>

* Merged vm_cache_ref and vm_cache to a single structure (Axel & Ingo).
* Renamed vm_cache.c to vm_cache.cpp


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21642 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 25f46ea4 30-Jun-2007 Axel Dörfler <axeld@pinc-software.de>

* Reserving areas never put down their reference of their address space when
removed in several cases.
* vfs_get_vnode_cache() now always gives out a reference to the cache it
returns; if it needs to allocate a new one, the vnode owns one reference,
and the caller another.
* therefore, file_cache_create() now owns a reference to its vm_cache_ref, and
frees it in file_cache_delete().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21533 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 245aecda 21-Jun-2007 Axel Dörfler <axeld@pinc-software.de>

Got rid of vnode_id and mount_id, replaced with ino_t and dev_t.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21485 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 70a11cec 02-Apr-2007 Axel Dörfler <axeld@pinc-software.de>

* pages_io() now handles it gracefully in case the fileVec array is too
small to hold the information for the requested I/O size.
* get_file_map() returned B_BUFFER_OVERFLOW already in case the array
was exactly as large as needed.
* read_chunk_into_cache() and write_chunk_to_cache() will no longer
override their local "size" variable.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20509 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 57a86098 31-Mar-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

A few more changes to pages_io():
* Added a few comments.
* Simplified the nested while loops by dropping the special handling for the
first iovec and restructuring the innermost loop. This also rules out
the possibility of a zero-length temporary vec. IMHO the readability
has improved quite a bit (YMMV :-). Hopefully without introducing new
bugs; please review!
* Corrected computation of totalSize in case less than size has been
read/written.
* Also set *_numBytes in case all fileVecs have been processed. Only
relevant in case the request extends beyond the end of file.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20483 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 061816ee 31-Mar-2007 Axel Dörfler <axeld@pinc-software.de>

* Fixed two bugs found by Ingo: the tempVec array bounds weren't checked when
filling them which could have written over the stack, and their iovec length
was set for the wrong iovec, potentially clobbering any memory.
* The first tempVec was usually empty, anyway, as the wrong iovec was chosen
to start from (usually one too early).
* The tempVec loop is now repeated until the whole fileVec is completed.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20476 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e4b4574f 11-Mar-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

Reordered somewhat unhealthy looking if-construct (first
"currentPage->state == ..." then "currentPage != NULL").


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20377 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c40fe37f 30-Aug-2006 Axel Dörfler <axeld@pinc-software.de>

* Reverted the changes r17693 made to vfs_get_vnode_cache(); instead, vm_create_vnode_cache()
will now grab a reference to the vnode as well if successful. This way, vfs_get_vnode_cache()
now actually works how it should: it will now always grab a reference to the cache and
its underlying vnode. This removes an extra reference to the vnode (and vm_cache) that
got ignored before and prevented volumes to be unmounted (or file caches to be removed).
Thanks to Korli for pointing this out.
* file_cache_create() is now aware of that extra vnode reference and releases it; unmounting
volumes is now working again as it should.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@18716 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 517dfdf4 22-May-2006 Axel Dörfler <axeld@pinc-software.de>

The cache did not correctly retrieve the file map when the file was fragmented
into more than MAX_FILE_IO_VECS pieces. This could have very weird consequences
like overwriting data outside the file (but on that same partition only).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@17536 a95241bf-73f2-0310-859d-f6bbb57e9c96


# bd5ed534 22-May-2006 Stephan Aßmus <superstippi@gmx.de>

a fix to the file cache (done by Axel), that prevents a crash when
reading the file map of certain files (hope I got that more or less
right)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@17525 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 97e06971 12-Apr-2006 Axel Dörfler <axeld@pinc-software.de>

Added a "reenter" parameter to the {read|write}_pages() functions to give file
systems a chance to know if they have locked already.
This fixes a locking problem in BFS where one thread tried to acquire two read
locks (where someone else trying to acquire a write lock would have caused a
dead lock).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@17108 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 8645479b 11-Apr-2006 Axel Dörfler <axeld@pinc-software.de>

* cache_io() could insert a second page at the same position in the vm_cache
since it unlocked the cache while waiting on a busy page. Now, we're filling
the pending request before unlocking the cache.
* Fixed the deadlock I mentioned in the last commit: if a page fault happens
at the same time we're trying to read/write from/to a page, we no longer
fight for the BFS inode lock, but eventually doing the job twice if needed.
Will need to go over the "write modified" functions to make sure they are
behaving as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@17091 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 8413da23 05-Mar-2006 Axel Dörfler <axeld@pinc-software.de>

Like the comment above said, vfs_lookup_vnode() doesn't grab a ref to the vnode,
so we better shouldn't release it.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16591 a95241bf-73f2-0310-859d-f6bbb57e9c96


# b2707997 26-Feb-2006 Stephan Aßmus <superstippi@gmx.de>

* read_chunk_into_cache() might not have put back physical pages
in case of a read error.

(coded by axeld)



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16510 a95241bf-73f2-0310-859d-f6bbb57e9c96


# d31d9974 07-Feb-2006 Axel Dörfler <axeld@pinc-software.de>

Slightly improved debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16288 a95241bf-73f2-0310-859d-f6bbb57e9c96


# ea2cd27e 21-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

* Fixed the strange vm_cache_ref reference count mechanism: now, a fresh
vm_cache_ref starts with a reference count of 1. When acquiring a vm_cache,
you no longer need to worry if that should go through the vm_store, or not;
as it now always does.
* map_backing_store() no longer needs to play with the vm_cache_ref
references.
* that simplified some code.
* vfs_get_vnode_cache() now grabs a reference to the cache, if successful.
* better balanced vnode ownership on vnode_store creation (vnode_store
released the vnode before if its creation failed).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15641 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 79f73dbc 20-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

* vm_page::offset is now called cache_offset and is now an uint32 instead of off_t;
this saves 4 bytes per page. To compensate the loss of bytes, the offset is now
stored in page size units, that's enough to address 2^44 or 16 TB (which is now
the maximal supported file size!).
* Renamed vm_page::ppn to physical_page_number.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15637 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 2b028fca 23-Nov-2005 Axel Dörfler <axeld@pinc-software.de>

Removed one TODO from the list: in case pages_io() fails, read_chunk_into_cache()
will no longer panic, but free its allocated pages.
I ran into this because BFS managed to create a file without data stream but
with a length larger than 0...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15093 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1c88c851 19-Oct-2005 Axel Dörfler <axeld@pinc-software.de>

Turned off debug output again - also temporarily removed the file cache warning
about our block_io module not honouring the total length in read_pages().
Removed drops into the debugger when there is a block without an "original"
data buffer - that's completely normal and happens when someone asks for
a cleared block that is not yet in the cache.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14437 a95241bf-73f2-0310-859d-f6bbb57e9c96


# b3373253 28-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

The generic syscall now returns more correct status codes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14078 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4784eca1 22-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

cache_io() did not set the state of a page to PAGE_STATE_MODIFIED when it wrote
to it - if that page was read from disk unmodified before (or written back in
the mean time), the updated contents coult not be detected, and therefore, were
never written back.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14048 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e6b68254 20-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

Fixed a stupid bug in the recently updated write_chunk_to_cache(): lastPageOffset
was incorrectly calculated, and could therefore cause all sorts of troubles with
writes over 4 kB.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14011 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 99e1c71e 20-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

- fixed a locking bug in the file cache: get_file_map() could been called with either
only a read lock on the inode, or without a lock at all (in case of prefetching) - it
is now using the cache_ref's lock to make sure it's retrieved only once and doesn't
waste/corrupts any memory.
- since that fix made it mandatory to fix write_chunk_into_cache() to not call pages_io()
while holding the cache_ref's lock (was a to-do item before), I changed it to make
that possible.
- It now also supports write-through caches in theory - as there is no way yet to tell
a cache to work this way.
- Optimized for the not so uncommon case of writing the last part of a file that is not
a multiple of the page size - it won't call pages_io() anymore then, but zero the rest
of the page directly.
- vm_page_write_modified() is now calling write_page() without holding the cache_ref's
lock as well.
The updated write_chunk_to_cache() is not so well tested, though, but appears to work
so far.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14006 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5913a657 17-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

Added two possible "ToDo" comments.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13970 a95241bf-73f2-0310-859d-f6bbb57e9c96


# b96ea69c 12-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

You really can do the same mistake twice, even in two subsequent lines of code...
get_file_map() now seems to work correctly in all regular cases.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13947 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 139353cf 12-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

Fixed two bugs in the new file extent caching code:
- under certain situations, the wrong extent could be chosen (off-by-one error)
- the resulting length of the first vector was incorrectly calculated for all
extents but the first
Improved error checking a bit.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13941 a95241bf-73f2-0310-859d-f6bbb57e9c96


# eab435cd 05-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

vfs_get_vnode_cache() now only allocates a new cache if requested: this
prevents the system to allocate caches for files that don't use or have
a file cache (ie. only those can be mmap()ed!).
Therefore, cache_prefetch() no longer crashes when trying to prefetch
files without a file cache.
read_into_cache() no longer does anything if the requested size is 0.
Fixed a bug in cache_prefetch_vnode(): if the cache couldn't be retrieved,
it put the vnode, but didn't own it (the caller does).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13904 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 95839f5c 03-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

The file cache now has another init method that is called after the boot
device becomes available.
Currently, it opens the "launch_speedup" module (if available), later it
should consult a settings file for what to do.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13894 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 11a3346c 01-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

The file extent map is now cached as well. Code is not optimal yet, and almost not
tested for fragmented files - but it seems to work good enough for now.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13870 a95241bf-73f2-0310-859d-f6bbb57e9c96


# b72cf260 01-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

Renamed vfs_vnode_acquire_ref() to vnode_acquire_vnode().
Removed vfs_vnode_release_ref(), as vfs_put_vnode() already does the same thing.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13867 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 18685752 01-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

cache_prefetch() now comes in two flavours: one with a direct vnode pointer,
the other one with usual device/inode ID pair.
Both versions now accept an offset/size pair to specify the region of the
file to be prefetched - this may be turned into a file_vec_io array later on.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13866 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c9955bf0 13-Jul-2005 Axel Dörfler <axeld@pinc-software.de>

Added new but empty function file_cache_invalidate_file_map().
This is needed in case the on-disk representation of a file changes (due to reorganization/defragmentation).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13660 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 2d690920 13-Apr-2005 Axel Dörfler <axeld@pinc-software.de>

Renamed system/core to system/kernel.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12360 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9c5c5990414f9570d5271328ce67252d50dfe8a2 15-Sep-2014 Paweł Dziepak <pdziepak@quarnos.org>

kernel: pagecache: provided buffers are not always in user memory

Source or destination buffers passed to pagecache functions may belong
to kernel memory (e.g. when the caller is packagefs). Because of that
we should tell vm_memcpy_{from, to}_physical() truth, not assume that all
buffers are in user memory. That's important because user memory page fault
handlers cannot be nested and these functions may be used while handling
a page fault.

With high probability fixes #11246.

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>


# 0e88a887b4a9ecaaf1062078d9ca9bfca78fcf3a 13-Jun-2012 Alex Smith <alex@alex-smith.me.uk>

First round of 64-bit safety fixes in the kernel.

* Most of this is incorrect printf format strings. Changed all strings
causing errors to use the B_PRI* format string definitions, which
means the strings should be correct across all platforms.
* Some other fixes for errors, casts required, etc.


# f8154d172da77bd77316f14c76d428bae7376323 02-Nov-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

mmlr (distracted) + bonefish:
* Turn VMCache::consumers C list into a DoublyLinkedList.
* Use object caches for the different VMCache types and the VMCacheRefs.
The purpose is to reduce slab area fragmentation.
* Requires the introduction of a pure virtual VMCache::DeleteObject()
method, implemented in the derived classes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@43133 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 222b8cf7b746f1f5406a21a1ca5b87458de6daf2 18-May-2011 Ingo Weinhold <ingo_weinhold@gmx.de>

cache_io(): Since satisfy_cache_io() (respectively the function it calls)
unlocks the cache, we have to look up the page again, since someone else could
have removed it in the meantime. Possibly fixes #7514.

+r1alpha3


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@41564 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1d578e15fe5b5c3ff62866ae81aef529d00d7762 02-Jun-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

Fixed more address types related issues. Mostly printf() or comparison
warnings, but also some oversights from earlier changes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@37000 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 435c43f5912b109e7d5cf682865d2061e62fad8c 02-Jun-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* Introduced type generic_io_vec, which is similar to iovec, but uses types
that are wide enough for both virtual and physical addresses.
* DMABuffer, IORequest, IOScheduler,... and code using them: Use
generic_io_vec and generic_{addr,size}_t where necessary.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36997 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4fe399e4262ea3b02d50fc3301415294665f5926 28-Apr-2010 Axel Dörfler <axeld@pinc-software.de>

* Use vm_page_write_modified_page_range() instead of
vm_page_schedule_write_page_range() to prevent more memory pressure.
* While it seems to help a bit, it doesn't solve the problem of bug #5777.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@36519 a95241bf-73f2-0310-859d-f6bbb57e9c96


# bd7645a12ab594d5eb1c6d6a70c3a82a429410dd 20-Feb-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* Made vm_page::state private and added accessor methods.
* Added kernel tracing for page state transitions.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35538 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 211c63df5ad40cfbd6aaba9b722166e6f6132e4e 19-Feb-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

reserve_pages(): When cannibalizing pages from the own cache it didn't check
whether the pages where mapped. Was introduced already in r22731.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35532 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9824ded0b01fa76d2dc9c4e7fb36d27a2e880446 20-Feb-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

cache_io(): Also requeue the last touched page to keep things roughly LRU
sorted.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35530 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1f611c018b474c78f84ea905e574c24586ae85b1 16-Feb-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

The file cache code was completely ignoring the vm_page::modified flag. In
particular it wouldn't set the flag when writing something to a page, but
only move it to the modified queue. Since mapping the page would move it to
another queue, the information that the page was modified would be lost and
it would never be written to disk. Was well reproducible with a Haiku image
build and limited amount of memory.
Fixes the hopefully last remaining cause for #5374.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35500 a95241bf-73f2-0310-859d-f6bbb57e9c96


# cee04e8074ec61fd8c6dac3539c5b821c3618888 08-Feb-2010 Artur Wyszynski <aljen-mlists@o2.pl>

Fixed various errors/warnings reported by cppcheck:
* memory leaks
* resource leaks
* added const's to getters
* removed a few reundant conditions


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35433 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 40bb94819e6c39d72ab29edc1a0dcd80b15b8b42 03-Feb-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* Removed useless return parameter from vm_remove_all_page_mappings().
* Added vm_clear_page_mapping_accessed_flags() and
vm_remove_all_page_mappings_if_unaccessed(), which combine the functionality
of vm_test_map_activation(), vm_clear_map_flags(), and
vm_remove_all_page_mappings(), thus saving lots of calls to translation map
methods. The backend is the new method
VMTranslationMap::ClearAccessedAndModified().
* Started to make use of the cached page queue and changed the meaning of the
other non-free queues slightly:
- Active queue: Contains mapped pages that have been used recently.
- Inactive queue: Contains mapped pages that have not been used recently. Also
contains unmapped temporary pages.
- Modified queue: Contains unmapped modified pages.
- Cached queue: Contains unmapped unmodified pages (LRU sorted).
Unless we're actually low on memory and actively do paging, modified and
cached queues only contain non-temporary pages. Cached pages are considered
quasi free. They still belong to a cache, but since they are unmodified and
unmapped, they can be freed immediately. And this is what
vm_page_[try_]reserve_pages() do now when there are no more actually free
pages at hand. Essentially this means that pages storing cached file data,
unless mmap()ped, no longer are considered used and don't contribute to page
pressure. Paging will not happen as long there are enough free + cached pages
available.
* Reimplemented the page daemon. It no longer scans all pages, but instead works
the page queues. As long as the free pages situation is harmless, it only
iterates through the active queue and deactivates pages that have not been
used recently. When paging occurs it additionally scans the inactive queue and
frees pages that have not been used recently.
* Changed the page reservation/allocation interface:
vm_page_[try_]reserve_pages(), vm_page_unreserve_pages(), and
vm_page_allocate_page() now take a vm_page_reservation structure pointer.
The reservation functions initialize the structure -- currently consisting
only of a count member for the number of still reserved pages.
vm_page_allocate_page() decrements the count and vm_page_unreserve_pages()
unreserves the remaining pages (if any). Advantages are that reservation/
unreservation mismatches cannot occur anymore, that vm_page_allocate_page()
can verify that the caller has indeed a reserved page left, and that there's
no unnecessary pressure on the free page pool anymore. The only disadvantage
is that the vm_page_reservation object needs to be passed around a bit.
* Reworked the page reservation implementation:
- Got rid of sSystemReservedPages and sPageDeficit. Instead
sUnreservedFreePages now actually contains the number of free pages that
have not yet been reserved (it cannot become negative anymore) and the new
sUnsatisfiedPageReservations contains the number of pages that are still
needed for reservation.
- Threads waiting for reservations do now add themselves to a waiter queue,
which is ordered by descending priority (VM priority and thread priority).
High priority waiters are served first when pages become available.
Fixes #5328.
* cache_prefetch_vnode(): Would reserve one less page than allocated later, if
the size wasn't page aligned.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35393 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e65c400299386f99a251395ff2e59572705d7e49 29-Jan-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* Replaced the vm_page_allocate_page*() "pageState" parameter by a more
general "flags" parameter. It encodes the target state of the page -- so
that the page isn't unnecessarily put in the wrong page queue first -- a
flag whether the page should be cleared, and one to indicate whether the
page should be marked busy.
* Added page state PAGE_STATE_CACHED. Not used yet.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35333 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 72382fa6291e810be2949a70abd8f274f92dbd2c 29-Jan-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* Removed the page state PAGE_STATE_BUSY and instead introduced a vm_page::busy
flag. The obvious advantage is that one can still see what state a page is in
and even move it between states while being marked busy.
* Removed the vm_page::is_dummy flag. Instead we mark marker pages busy, which
in all cases has the same effect. Introduced a vm_page_is_dummy() that can
still check whether a given page is a dummy page.
* vm_page_unreserve_pages(): Before adding to the system reserve make sure
sUnreservedFreePages is non-negative. Otherwise we'd make nonexisting pages
available for allocation. steal_pages() still has the same problem and it
can't be solved that easily.
* map_page(): No longer changes the page state/mark the page unbusy. That's the
caller's responsibility.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35331 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 64a9bd15c856a4da7d017294df73dc4d46350e9b 26-Jan-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

Optimized writing of zeroes to files. Instead of allocating a small buffer
on the fly, clearing and writing it each time, we now use an iovec with 32
identical entries pointing to a clear page that we prepare once at
initialization. This speeds up clear_image in low memory situations
dramatically.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35304 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 7fa3e4e68a17d41ffa9b2ce945d5aeec6ddf7e61 26-Jan-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

cache_io(): Reevaluate the memory situation periodically. Deciding at the
beginning whether to pass the cache by really doesn't help when
reading/writing a huge amount of data, since a low memory situation is likely
to occur at some point during the operation. This should fix the main issue
of #3768.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35299 a95241bf-73f2-0310-859d-f6bbb57e9c96


# cff6e9e406132a76bfc20cb35ff5228dd0ba94d8 26-Jan-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* The system now holds back a small reserve of committable memory and pages. The
memory and page reservation functions have a new "priority" parameter that
indicates how deep the function may tap into that reserve. The currently
existing priority levels are "user", "system", and "VIP". The idea is that
user programs should never be able to cause a state that gets the kernel into
trouble due to heavy battling for memory. The "VIP" level (not really used
yet) is intended for allocations that are required to free memory eventually
(in the page writer). More levels are thinkable in the future, like "user real
time" or "user system server".
* Added "priority" parameters to several VMCache methods.
* Replaced the map_backing_store() "unmapAddressRange" parameter by a "flags"
parameter.
* Added area creation flag CREATE_AREA_PRIORITY_VIP and slab allocator flag
CACHE_PRIORITY_VIP indicating the importance of the request.
* Changed most code to pass the right priorities/flags.

These changes already significantly improve the behavior in low memory
situations. I've tested a bit with 64 MB (virtual) RAM and, while not
particularly fast and responsive, the system remains at least usable under high
memory pressure.
As a side effect the slab allocator can now be used as general memory allocator.
Not done by default yet, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@35295 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3cd2094396dde9ca42263c535041a95d5f0d5fff 06-Jan-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* Added new debug feature (DEBUG_PAGE_ACCESS) to detect invalid concurrent
access to a vm_page. It is basically an atomically accessed thread ID field
in the vm_page structure, which is explicitly set by macros marking the
critical sections. As a first positive effect I had to review quite a bit of
code and found several issues.
* Added several TODOs and comments. Some harmless ones, but also a few
troublesome ones in vm.cpp regarding page unmapping.
* file_cache: PrecacheIO::Prepare()/read_into_cache: Removed superfluous
vm_page_allocate_page() return value checks. It cannot fail anymore.
* Removed the heavily contended "pages" lock. We use different policies now:
- sModifiedTemporaryPages is accessed atomically.
- sPageDeficitLock and sFreePageCondition are protected by a new mutex.
- The page queues have individual locks (mutexes).
- Renamed set_page_state_nolock() to set_page_state(). Unless the caller says
otherwise, it does now lock the affected pages queues itself. Also changed
the return value to void -- we panic() anyway.
* set_page_state(): Add free/clear pages to the beginning of their respective
queues as this is more cache-friendly.
* Pages with the states PAGE_STATE_WIRED or PAGE_STATE_UNUSED are no longer
in any queue. They were in the "active" queue, but there's no good reason
to have them there. In case we decide to let the page daemon work the queues
(like FreeBSD) they would just be in the way.
* Pulled the common part of vm_page_allocate_page_run[_no_base]() into a helper
function. Also fixed a bug I introduced previously: The functions must not
vm_page_unreserve_pages() on success, since they remove the pages from the
free/clear queue without decrementing sUnreservedFreePages.
* vm_page_set_state(): Changed return type to void. The function cannot really
fail and no-one was checking it anyway.
* vm_page_free(), vm_page_set_state(): Added assertion: The page must not be
free/clear before. This is implied by the policy that no-one is allowed to
access free/clear pages without holding the respective queue's lock, which is
not the case at this point. This found the bug fixed in r34912.
* vm_page_requeue(): Added general assertions. panic() when requeuing of
free/clear pages is requested. Same reason as above.
* vm_clone_area(), B_FULL_LOCK case: Don't map busy pages. The implementation is
still not correct, though.

My usual -j8 Haiku build test runs another 10% faster, now. The total kernel
time drops about 18%. As hoped the new locks have only a fraction of the old
"pages" lock contention. Other locks lead the "most wanted list" now.



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34933 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1021fd28262697dbbbe1d54a868f0672900c78f3 01-Jan-2010 Ingo Weinhold <ingo_weinhold@gmx.de>

* agp_gart(): Use vm_page_[un]reserve_pages().
* Removed unused vm_page_allocate_pages().
* Removed now unused (always true) "reserved" parameter from
vm_page_allocate_page().
* Removed unused (always false) "stealActive" parameter from steal_page().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34836 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 522c2f19d458245474fcaf5b0254e0099906b117 07-Dec-2009 Ingo Weinhold <ingo_weinhold@gmx.de>

* Added a simple mechanism to wait for events to VMCache. WaitForPageEvents()
waits for certain events on a given page, NotifyPageEvents() wakes up
waiting threads respectively.
* Used the new feature instead of condition variables for waiting on busy
pages. We save publishing and unpublishing of a condition variable whenever
a page is marked busy. There's only something to do, if there's at least
one thread waiting in the list of the respective cache. The general
assumption is that this is only rarely the case and even if it happens,
there should be only very few threads.
* Added an apparently missing notification in cache_io(). At least I didn't
see the reason for it not being there.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34537 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e50cf8765be50a7454c9488db38b638cf90805af 02-Dec-2009 Ingo Weinhold <ingo_weinhold@gmx.de>

* Moved the VM headers into subdirectory vm/.
* Renamed vm_cache.h/vm_address_space.h to VMCache.h/VMAddressSpace.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34449 a95241bf-73f2-0310-859d-f6bbb57e9c96


# a477e3cf200097774f88ed4e4134f87674a2a063 01-Dec-2009 Ingo Weinhold <ingo_weinhold@gmx.de>

Finished renaming of vm_cache to VMCache.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@34424 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1af7d115046ccc76641973fe434ed5760e6fdd20 10-Oct-2009 Michael Lotz <mmlr@mlotz.ch>

* Rework page writing to combine page writes where possible. For now the pages
are required to be physically contiguos, which should be reworked to put them
into seperate iovecs. Still this manages to combine a great deal of page
writes into larger bursts already. Reduces the amount of IO requests being
scheduled (and greatly benefits media where page wise writes are slow when
they are accessed through a non-IOScheduler path, i.e. USB mass storage until
that is properly implemented).
* Abstracted per page page writing tasks into a PageWriteWrapper class.
* Abstracted per transfer page writing tasks into PageWriteTransfer class which
formerly was the PageWriterCallback.
* Use both classes from the PageWriterRun and from
vm_page_write_modified_page_range to remove code duplication.
* Adjusted synchronous VMAnonymousCache::Write() to cope correctly with larger
iovecs and more than one iovec. It assumed that there was exactly one page per
vector previously.
* Introduced MaxPagesPerWrite() and MaxPagesPerAsyncWrite() to VMCache to allow
a cache to specify restricitions. VMAnonymousCache does restrict the max pages
to 1 for WriteAsync right now as I didn't feel like reworking that one to cope
with non single page writes just yet.
* Pulled out PageWriteTransfer methods for better readability.
* Some typo fixes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33507 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 61b6f38cfc5aca5235a33f1815595886c49d3b18 31-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* Cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@32021 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c32499b4f7f934d05e1d4f390f68267803ae246b 30-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* Since the file_map reports its chunk with a file system specific alignment, we
cannot assume that the file is that long - we need to check for the actual
file size specifically, and then clear the extraneous bytes based on that one.
* This fixes seeing old file left-overs in the space beyond the file size in
mmapped file. Thanks to Oliver who provided me with a nice test program to
reproduce this problem (this should also fix gcc's fix_includes app (although
I wonder what it does if a file actually ends on a page boundary).
* Also fixed a bug in precaching that would cause the last page of the cache
to be discarded, causing it to be re-read later on.
* Moved partial page clearing into its own function
read_pages_and_clear_partial(), even though it's currently only used once.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@32001 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 6102f0788a4204b76d177595bd48eda06fcfb9d8 24-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* file_cache_set_size() now clears the last page of its cache if needed. This
fixes #4106 for good.
* Also removed a bit of dead code in that function.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31732 a95241bf-73f2-0310-859d-f6bbb57e9c96


# db8fb4fdf024dbe77db8cc7d56296e35d9974584 21-Jul-2009 Axel Dörfler <axeld@pinc-software.de>

* The file cache must clear partial pages as well, since they can be mapped
directly via mmap().
* This fixes bug #4106.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@31669 a95241bf-73f2-0310-859d-f6bbb57e9c96


# a5dbd78b7af2bb7ca38be351331943ef4bc0089e 05-May-2009 Axel Dörfler <axeld@pinc-software.de>

* Since PrecacheIO is used as asynchronous callback object, we must not access
it anymore after having called vfs_asynchronous_read_pages().
* Now, Prepare() does all the preparation work, and ReadAsync() does the actual
work - this must be called without having the cache locked. This also fixes
another bug where the callback would be deleted twice in case the I/O request
failed.
* This fixes bug #3847.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30632 a95241bf-73f2-0310-859d-f6bbb57e9c96


# d5ad762913429ae34478222a0c87199be0709439 30-Apr-2009 Axel Dörfler <axeld@pinc-software.de>

Fixed several problems of the prefetching code:
* Did claim to have reserved pages when calling vm_page_allocate_page(), but
didn't have any (copy&paste bug). We cannot use it without reserved pages,
as we need to call vm_page_allocate_page() with a cache locked.
* No longer use low_resource_state() to determine whether to precache or not,
but use the new vm_page_num_used_pages() instead.
* Also don't (try to) precache when the cache already has more than 2/3 of its
pages to safe some unnecessary work.
* The size to precache was limited to the file size incorrectly.
* When precaching failed, the cache reference was not released.
* The precaching started one page too late, causing bug #3835.
* Reenabled precaching.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30515 a95241bf-73f2-0310-859d-f6bbb57e9c96


# aff0fbbe57480a2d6cc6f375444e760c838abbab 28-Apr-2009 Stephan Aßmus <superstippi@gmx.de>

axeld:
* vfs_read_pages_async() must not be called with locked cache,
since it cannot be guaranteed that the operation will be
performed asynchronously. (The ISO9660 FS for example does not
implement the new IO hooks... yet.)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30476 a95241bf-73f2-0310-859d-f6bbb57e9c96


# eb2bd0e8e3689998717f91d8b9023296e3f32e78 27-Apr-2009 Stephan Aßmus <superstippi@gmx.de>

axeld:
* Implemented a way to do asynchronous pre-fetching when mapping files.
* There are slight code duplications in some places that could benefit
from cleaning up, but nothing too bad.
* Implementing smarter ways to trigger prefetching and more analysis of
the situations in the kernel would be nice. Currently up to 10 MB
of every mapped file are pre-fetched without further analysis.
* The speed improvement is nice for certain operations. On our test
system (real hardware), Firefox took 9 seconds from being launched
to display a window. Now it takes 5 seconds. Both measurements
right after booting. The same system took 35 seconds from launching
Haiku in the GRUB menu to displaying the Tracker desktop background
image. Now it takes 27 seconds.
* We didn't have the chance to check out the effects of this on the
CD boot, but potentially, they could speed it up a lot.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30465 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 8a26f35a6d174a8a7a8f8846fa7f5449cad28a2a 20-Apr-2009 Axel Dörfler <axeld@pinc-software.de>

* Sequential write accesses were never detected, due to an incorrect check.
* The previous code would have scheduled a single page to be written out (if it
would have ever been triggered), now we schedule the complete previous write
access. This greatly speeds up a "dd if=/dev/zero of=test ..." beyond the
size of available memory.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@30276 a95241bf-73f2-0310-859d-f6bbb57e9c96


# aa4ba93e25c1c63730ba69e04d3d96c3253924fd 08-Mar-2009 Ingo Weinhold <ingo_weinhold@gmx.de>

* Renamed src/system/kernel/device_manager/io_requests.{h,cpp} to
IORequest.{h,cpp}.
* Introduced public <io_requests.h> header. Currently it only declares the
single function BFS uses.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@29446 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 47c40a10a10dc615e078754503f2c19b9f98c38d 19-Oct-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* Prefixed memset_physical() and memcpy_to_physical() with "vm_",
added vm_memcpy_from_physical() and vm_memcpy_physical_page(), and
added respective functions to the vm_translation_map operations. The
architecture specific implementation can now decide how to implement
them most efficiently. Added generic implementations that can be used,
though.
* Changed vm_{get,put}_physical_page(). The former no longer accepts
flags (the only flag PHYSICAL_PAGE_DONT_WAIT wasn't needed anymore).
Instead it returns an implementation-specific handle that has to be
passed to the latter. Added vm_{get,put}_physical_page_current_cpu()
and *_debug() variants, that work only for the current CPU,
respectively when in the kernel debugger. Also adjusted the
vm_translation_map operations accordingly.
* Made consequent use of the physical memory operations in the source
tree.
* Also adjusted the m68k and ppc implementations with respect to the
vm_translation_map operation changes, but they are probably broken,
nevertheless.
* For x86 the generic physical page mapper isn't used anymore. It is
suboptimal in any case. For systems with small memory it is too much
overhead, since one can just map the complete physical memory (that's
not done yet, though). For systems with large memory it counteracts
the VM strategy to reuse the least recently used pages. Since those
pages will most likely not be mapped by the page mapper anymore, it
will keep remapping chunks. This was also the reason why building
Haiku in Haiku was significantly faster with only 256 MB RAM (since
that much could be kept mapped all the time).
Now we're using a different strategy: We have small pools of virtual
page slots per CPU that are used for the physical page operations
(memset_physical(), memcpy_*_physical()) with CPU-pinned thread.
Furthermore we have four slots per translation map, which are used to
map page tables.

These changes speed up the Haiku image build in Haiku significantly. On
my Core2 Duo 2.2 GHz 2 GB machine about 40% to 20 min 40 s (KDEBUG
disabled, block cache debug disabled). Still more than factor 3 slower
than FreeBSD and Linux, though.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28244 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 7b8683b2524b1f17bbeb53eddbe7eeda48898df5 17-Oct-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Updated write_to_cache() to use physical pages for I/O and the new
{memset,memcpy_to}_physical() functions.
Mapping lots of physical pages at once as done before was an actual
problem on systems with enough RAM, as the physical page mapper can map
only 64 chunks at a time. So multiple threads could play dining
philosophers, each getting only one of two chopsticks, waiting for
another one to be freed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@28220 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1b6eff280f4afcf4d7c9dc9ccdc3a65f4e6ca0fd 11-Oct-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* Replaced the vm_get_physical_page() "flags"
PHYSICAL_PAGE_{NO,CAN}_WAIT into an actual flag
PHYSICAL_PAGE_DONT_WAIT.
* Pass the flags through to the chunk mapper callback.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@27979 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3c1a3047a49f116cf5ce5696a44a635a85dc9f15 04-Aug-2008 Axel Dörfler <axeld@pinc-software.de>

* Added file_cache_is_enabled() function.
* Added file_map_set_mode() function that you can use to keep a whole file
cached. This is needed for the swap file support: FILE_MAP_CACHE_ALL will
not only precache all file_io_vecs when called, but it will also cause all
file_map_translate() calls to fail that would require further caching (ie.
if the file size had changed).
* Updated the fs_shell file map code to the latest one (with several bug fixes).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26785 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 7491000f2098a73772a5914dc4c4bd62ffd616da 03-Aug-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Added functions file_cache_{disable,enable}(). They allow to disable
actual caching in the file cache, i.e. all reads and writes go directly
to the underlying device. The implementation is not quite complete,
since the VM can still add pages to the cache when the file is mmap()ed,
which can lead to inconsistencies.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26779 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9a0fc9a874cdc1b1f692630c7b13a519a184d35b 31-Jul-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

In cache_io(): Don't keep the cache locked while doing a
user_{memcpy,memset}(), since that can cause a page fault, which needs
pages and might try to steal some from our cache.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26700 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 7f12cc54a729622cd04940ee9400958413d99b21 30-Jul-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* It is now supported that I/O operations and requests are only handled
partially (e.g. due to hitting the end of file). The respective
classes have grown new methods and attributes to deal with that. The
"finished" callbacks have got additional parameters to indicate
whether the transfer was only partial and how much has been
transferred. Other callbacks and functions have a size_t* in/out
parameter instead of a simple size_t, now.
* vfs_{read,write}_pages() do now use the I/O request framework instead
of the underlying FS's {read,write}_pages() hooks (those should be
unused now). Furthermore they've got an additional "flags" parameter,
which is passed to IORequest::Init(), i.e. it allows to specify that
the given vecs refer to physical addresses.
* The file cache's read_into_cache() reads directly into physical
pages, now.
* Fixed bug in DoIO::IO(): The offset was not adjusted, so that all
pages were incorrectly transferred from/to the same location.
* Fixed broken subrequest scheduling loop head in
do_iterative_fd_io_iterate().
* Adjusted the test driver and implemented its io() hook. Using this
driver I/O requests are passed all the way from the VFS/VM to the
driver and through the I/O scheduler. It even seems to work. :-)
* Added missing const to the iovec* parameter of the IORequest::Init()
methods.
* Disabled some debug output by default. Added new optional debug
output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26692 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e6bd90c58dbae64f3b464edcff90dcb06e63a716 23-Jul-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* bfs_fsync() was the only place which could cause the
fs_vnode_ops::write_pages() to be called with fsReenter = true. Since
this is no longer the case, the argument has become superfluous. For
read_pages() it always was. Removed the argument from the functions
and all functions that propagated it.
* Some whitespace at the end of lines was removed.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26579 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 190712ced90f4499fefd9b5cf416a050f57a7946 23-Jul-2008 Axel Dörfler <axeld@pinc-software.de>

* file_cache_sync() no longer needs the file system's lock.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26577 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5c99d639708df9b4e2cc847b38d510149d19ec78 22-Jul-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Merged branch haiku/branches/developer/bonefish/vm into trunk. This
introduces the following relevant changes:
* VMCache:
- Renamed vm_cache to VMCache, merged it with vm_store and made it a
C++ class with virtual methods (replacing the store operations).
Turned the different store implementations into subclasses.
- Introduced MergeStore() callback, changed semantics of Commit().
- Changed locking and referencing semantics. A reference can only be
acquired/released with the cache locked. An unreferenced cache is
deleted and a mergeable cache merged when it is unlocked. This
removes the "busy" state of a cache and simplifies the page fault
code.
* Added VMAnonymousCache, which will implement swap support (work by
Zhao Shuai). It is not integrated and used yet, though.
* Enabled the mutex/recursive lock holder asserts.
* Fixed DoublyLinkedList::Swap().
* Generalized the low memory handler to a low resource handler. And made
semaphores and reserved memory handled resources. Made
vm_try_resource_memory() optionally wait (with timeout), and used that
feature to reserve memory for areas.
...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26572 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 13ea2be597b78a3a0e8eeec607fe184da0326c98 16-Jul-2008 Axel Dörfler <axeld@pinc-software.de>

* Since cache_io() can now work with NULL buffers, I resurrected prefetching
using it.
* IOW cache_prefetch_vnode() should work again now.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26468 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9e1ea0e7d4893aabf98b0d8a2c20b48cf997fbd2 28-Jun-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Don't leak the buffer allocated at the beginning of the function.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26161 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e1b630c55d9d0c25aef812efc9c2410f5ba53456 28-Jun-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* Replaced the global cache pages hash table by an IteratableSplayTree
per cache.
* Changed the strategy vm_cache_acquire_page_cache_ref() uses to ensure
that the cache isn't deleted while trying to get a reference. Instead
of the global cache pages hash table lock, it holds the global cache
list lock now. We acquire + release this lock in delete_cache() after
removing all pages and just before deleting the object.
* Some small optimizations using the property that the cache's pages are
ordered, now (vm_cache_resize(), vm_page_write_modified_page_range(),
vm_page_schedule_write_page_range()).
* Replaced some code counting a cache's pages by simply using
vm_cache::page_count.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26160 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 0c615a01ae49634aaf59fbe35b3d55b3bb8890df 01-May-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* Removed old mutex implementation and renamed cutex to mutex.
* Trivial adjustments of code using mutexes. Mostly removing the
mutex_init() return value check.
* Added mutex_lock_threads_locked(), which is called with the threads
spinlock being held. The spinlock is released while waiting, of
course. This function is useful in cases where the existence of the
mutex object is ensured by holding the threads spinlock.
* Changed the two instances in the VFS code where an IO context of
another team needs to be locked to use mutex_lock_threads_locked().
Before it required a semaphore-based mutex implementation.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25283 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 184de764fe4d2593164db6e8b3d3a42021f6bc2f 30-Apr-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

Replaced the vm_cache mutex by a cutex. This should save quite a few
semaphores.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25277 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 6cef245eca821584f07f5a13558f51ec586852e8 22-Apr-2008 Ingo Weinhold <ingo_weinhold@gmx.de>

* Detemplatized ConditionVariable{Entry}. Merged them with their
respective Private* base class.
* Changed sigwait() and sigsuspend() to use thread_block() instead of a
condition variable.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25100 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c12ed8563a84ef1f4a67d3145dd4e17a00b7594a 25-Mar-2008 Axel Dörfler <axeld@pinc-software.de>

Fixed warning.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24581 a95241bf-73f2-0310-859d-f6bbb57e9c96


# a121b8c83ceba34954e7c187e84c206539207b3c 24-Mar-2008 Axel Dörfler <axeld@pinc-software.de>

* Added a new flag for write stat: B_STAT_SIZE_INSECURE that allows a
file system to not fill newly created space with zeros.
BFile::SetSize() now uses this, while [f]truncate() does not. This
is only a temporary work-around until BFS supports sparse files.
* Apps that want to reserve space to fill up later should use
BFile::SetSize() for now, as this will be a lot faster than
[f]truncate().
* cache_io() and the functions below now use a special mode when you
pass in a NULL buffer: for read access, the cache is only populated
(useful for prefetching), for write access, the file is filled with
zeros.
* Implemented BFS's Inode::FillGapWithZeros() using this method now.
* Removed extraneous white space.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@24555 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9ff70e7468b6408344532c9fbf9ea82701cc62df 26-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

read_from_file() and write_to_file() did not take the pageOffset into account.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22998 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3d268eda3d0ca504c865533347decf27b54025b6 10-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

* Extracted file_map API out of the file cache - it's now an optional service
that can be used by file systems.
* Changed the way the file cache works: instead of reading/writing to the
underlying device directly, it can now be used for any data source, ie.
also network file systems.
* As a result, the former pages_io() moved to the VFS layer, and can now be
called by a file system via {read|write}_file_io_vec_pages() (naming
suggestions are always welcomed :-)). It now gets an FD, and uses that to
communicate with the device (via its fs_{read|write}_pages() hooks).
* The file_cache_{read|write}() functions must now be called without holding
an I/O relevant file system lock. That allows the file cache to prepare the
pages without colliding with the page writer, IOW the "mayBlock" flag can
go into the attic again (yay!).
* This also results in a much better performance when the system does I/O and
is low on memory, as the page writer can now finally write back some pages,
and that even without maxing out the CPU :)
* The API changes put slightly more burden on the fs_{read|write}_pages()
hooks, but in combination with the file_map it's still pretty straight
forward. It just will have to dispatch the call to the underlying device
directly, usually it will just call its fs_{read|write}_pages() hooks
via the above mentioned calls.
* Ported BFS and FAT to the new API, the latter has not been tested, though.
* Also ported the API changes to the fs_shell. I also completely removed its
file cache level page handling - the downside is that device access is no
longer cached (ie. depends on the host OS now), the upside is that the code
is greatly simplified.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22886 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 9edc2b526f673a66ffbb312407bf093c2b84fbd5 08-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

The reserved pages computation used the same broken logic as read_into_cache()
and write_to_cache() before, IOW the cache could reserve too few pages.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22859 a95241bf-73f2-0310-859d-f6bbb57e9c96


# cfe386c2d5c4d83b121714dde8c05f40ed241dfb 08-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

* read_into_cache() and write_to_cache() both did not compute the number of
needed pages correctly, and would also not read/write enough in case the
offset didn't start at 0 resulting in undetected short reads/writes. It's
amazing how many bugs can be hidden in a few lines of code.
* Fixed a bug that might have been the cause for bug #1601: when the last part
of the write did not end on a page boundary, the last page had to be read
first, but that was done from the wrong offset. Also, if only parts of that
page could be read (because the file size didn't span over the whole page)
the remaining parts needed to be cleared.
* The cache_funcs were always called with the same value for numBytes and
bufferSize so I've eliminated the former.
* Large reads now also bypass the cache in case of low memory, large writes now
also only bypass the cache in that case, following Ingo's suggestion.
* Fixed compilation with debugging turned on.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22858 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 09149281e0a088a2a9ad78ea5abd033217c15bb8 08-Nov-2007 Axel Dörfler <axeld@pinc-software.de>

The file cache now completely bypasses the cache for writes equal or larger
than 64KB. Reads should probably get a similar logic, at least if memory is
tight.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22857 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c657332910e3873999704dd3eb151a3aff3efde9 26-Oct-2007 Axel Dörfler <axeld@pinc-software.de>

* The file cache now remembers the last 3 accesses to be able to detect
sequential access.
* This is now used to let the cache free some pages (or schedule them to be
written back) before vm_reserve_pages() is called, but only if that cache
is not mapped, and there is a memory shortage.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22731 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 0d871d3c5f090e106c23d4ca457dd83112ad3ed2 08-Oct-2007 Axel Dörfler <axeld@pinc-software.de>

* satisfy_cache_io() was using the wrong value to compute the number of
pages to reserve, which could result in to few being reserved in certain
situations.
* Use MutexLocker where appropriate.
* Reordered includes following the new rules.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22494 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 0710d59c1c6edace8ad333b1e3e86f9931e4e3f0 08-Oct-2007 Axel Dörfler <axeld@pinc-software.de>

* Added a "canWait" argument to vfs_get_vnode() - you can use this to ignore
busy vnodes.
* dir_create_entry_ref() used get_vnode() incorrectly (and could therefore
potentially prevent a file system from doing proper locking when called
from the kernel).
* The vnode_store now uses this for its acquire_unreferenced_ref()
implementation (and therefore for the page writer).
* read_into_cache() and write_to_cache() were still marked inline.
* The system will now wait 10 secs for a busy vnode before returning an error.
* It will also no longer panic in that case.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22485 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 20b232e900921a97e6b88917c04ca6777da551cd 07-Oct-2007 Axel Dörfler <axeld@pinc-software.de>

Actually forgot this nail: the file cache will now reserve the pages it will
allocate with the vm_cache locked - this is necessary to be able to steal pages
from itself (large files...).
The system doesn't actually lock up anymore, but it still renders itself unusable;
obviously the page thief does not work correctly, yet. The rest of the experience
is created by our current scheduler (the page thief runs and runs, but it doesn't
free any pages anymore).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22466 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 0633dcc2c695d57194204517f2d5486b8b27a821 06-Oct-2007 Axel Dörfler <axeld@pinc-software.de>

* Fixed a serious oversight that could result in trying to insert a page
twice into the same cache: cache_io() called read_into_cache() (or
write_to_cache()), and that broke down the request into smaller parts.
It then called read_chunk_into_cache() (or write_chunk_to_cache() resp.)
to actually allocate pages and fulfill the request.
However, it needed to unlock the cache for each chunk, and in the mean
time someone else could insert pages into the remaining chunks.
* Now, cache_io() already takes care of chunking the data which makes this
approach safe, and also simplified the code a bit - read_into_cache()/
write_to_cache() are gone now. I've renamed read_chunk_into_cache() to
read_into_cache() (same for the write function).
* Also got rid of that goto in that function while I was on it.
* Disabled cache_prefetch_vnode() for now (it's similar to cache_io(), but
since it's currently not used [since no cache module is installed yet],
I didn't want to go through updating it now, too).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22465 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 80f54692912d728f5780e9d78d7643828edaaa2a 06-Oct-2007 Axel Dörfler <axeld@pinc-software.de>

* struct vnode is an opaque type now, removed void* where it was used incorrectly.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22461 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 3e9513aa4a8bf5f2fb2b86a767ffe46f6e626acc 03-Oct-2007 Axel Dörfler <axeld@pinc-software.de>

* fs_{write|read}_pages() now has an additional argument "mayBlock".
* the page writer don't allow to block, while all other writers do. This fixes
bug #1509. The reason the page writer needs this is because it marks several
pages from different caches as busy.
* Fixed a warning about ASSERT being defined already in BFS, since
util/DoublyLinkedList.h now includes debug.h.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22434 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 44e2fd093c9a2374c0326e48e386f4dcf4a19568 26-Sep-2007 Axel Dörfler <axeld@pinc-software.de>

Actually, it's not a good idea to increase the usage_count, since every access
is tracked while the page scanner runs much slower. Now, we just set it to a
fixed positive, so that they can easily age in case they are actually unused.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22320 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 8e0f884c71025810db35b54234f1a2919131b108 26-Sep-2007 Axel Dörfler <axeld@pinc-software.de>

* Since the page scanner and thief can work more effectively when no vm_caches
are locked, there is now a vm_page_reserve_pages() call to ensure upfront that
there is a page for me when I need it, and may have locked some caches.
* The vm_soft_fault() routine now makes use of that feature.
* vm_page_allocate_page() now resets the vm_page::usage_count, so that the file
cache does not need to do this in read_chunk_into_cache() and
write_chunk_to_cache().
* In cache_io() however, it need to update the usage_count - and it does that
now. Since non-mapped caches don't have mappings, the page scanner will punish
the cache pages stronger than other pages which is accidently just what we
want.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22319 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 6d4aea47962e0ea08b81327fbbf93b32ae821177 25-Sep-2007 Axel Dörfler <axeld@pinc-software.de>

bonefish+axeld:
* Removed the vm_cache/vm_store ref_count duality that besides being a bit ugly
also created the page dameon cache retrieval problem: now, only areas (and
cache consumers) retrieve a reference to the store (and therefore, the vnode).
The page daemon doesn't need to care about this at all anymore, and the pseudo
references of the vm_cache could be removed again.
* Rearranged deletion of vnodes such that its ID can be reused directly after
fs_remove_vnode() has been called.
* vm_page_allocate_page() no longer panics when it runs out of pages, but just
waits for new pages to become available using the new sFreeCondition condition
variable - to make sure this happens in an acceptable time frame, it'll
trigger a run of the low memory handlers.
* Implemented a page_thief() that steals inactive pages from caches and puts
them into the free queue. It runs as a low memory handler.
* The file cache now sets the usage count on the pages it inserts into the
cache (needs some rework though, cache_io() doesn't do it yet).
* Instead of panicking, the kernel will currently dead lock in low memory
situations, since BFS does a bit too much in bfs_release_vnode().
* Some minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22315 a95241bf-73f2-0310-859d-f6bbb57e9c96


# b50494aaac8940817dd5c8789c1340fc2dba4887 02-Sep-2007 Axel Dörfler <axeld@pinc-software.de>

* file_cache_set_size() now calls file_cache_invalidate_file_map() with the
correct values (resolving a TODO); however, the latter doesn't do anything
with those yet.
* Cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@22154 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 13542314ef463942894f0ddc1d345bc03d25cd70 10-Aug-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

Removed special dummy page handling. It became obsolete with r21816,
since fault_find_page() does no longer insert a dummy page into a cache
that has a store from which it can read the page.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21887 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 279c6b76dc2a862f712eb2222381988f74b7b199 09-Aug-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

* Use condition variables when waiting for busy pages or busy caches.
* Removed a few instances where the page state was set busy directly after
allocating it. This is a no-op, since a page is always busy after
allocation.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21875 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 7c3a45ec5e2387c4a2aa68c6f8bd39f8655db775 03-Aug-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

Although probably not used ATM, cache_prefetch_vnode() should respect
busy pages.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21815 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 58f6e8e5e4aeee797ecd3bb7bed17d5a3a2029c0 17-Jul-2007 Axel Dörfler <axeld@pinc-software.de>

* Merged vm_cache_ref and vm_cache to a single structure (Axel & Ingo).
* Renamed vm_cache.c to vm_cache.cpp


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21642 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 25f46ea449ca29e08bef64ef93eb5d5b117d7e2f 30-Jun-2007 Axel Dörfler <axeld@pinc-software.de>

* Reserving areas never put down their reference of their address space when
removed in several cases.
* vfs_get_vnode_cache() now always gives out a reference to the cache it
returns; if it needs to allocate a new one, the vnode owns one reference,
and the caller another.
* therefore, file_cache_create() now owns a reference to its vm_cache_ref, and
frees it in file_cache_delete().


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21533 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 245aecda8ac43fc2c0c0bac6a7e4016efa9b71e1 21-Jun-2007 Axel Dörfler <axeld@pinc-software.de>

Got rid of vnode_id and mount_id, replaced with ino_t and dev_t.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@21485 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 70a11cecbdfc4d7e0f15746b8367d6efcedf95e9 02-Apr-2007 Axel Dörfler <axeld@pinc-software.de>

* pages_io() now handles it gracefully in case the fileVec array is too
small to hold the information for the requested I/O size.
* get_file_map() returned B_BUFFER_OVERFLOW already in case the array
was exactly as large as needed.
* read_chunk_into_cache() and write_chunk_to_cache() will no longer
override their local "size" variable.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20509 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 57a860987aeed12fbb132af17d3625adbb6bf570 31-Mar-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

A few more changes to pages_io():
* Added a few comments.
* Simplified the nested while loops by dropping the special handling for the
first iovec and restructuring the innermost loop. This also rules out
the possibility of a zero-length temporary vec. IMHO the readability
has improved quite a bit (YMMV :-). Hopefully without introducing new
bugs; please review!
* Corrected computation of totalSize in case less than size has been
read/written.
* Also set *_numBytes in case all fileVecs have been processed. Only
relevant in case the request extends beyond the end of file.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20483 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 061816eefa055bc2455218be007970db1db8a8b7 31-Mar-2007 Axel Dörfler <axeld@pinc-software.de>

* Fixed two bugs found by Ingo: the tempVec array bounds weren't checked when
filling them which could have written over the stack, and their iovec length
was set for the wrong iovec, potentially clobbering any memory.
* The first tempVec was usually empty, anyway, as the wrong iovec was chosen
to start from (usually one too early).
* The tempVec loop is now repeated until the whole fileVec is completed.
* Minor cleanup.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20476 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e4b4574f41be26f5ae3814d171e674178ebae0fc 11-Mar-2007 Ingo Weinhold <ingo_weinhold@gmx.de>

Reordered somewhat unhealthy looking if-construct (first
"currentPage->state == ..." then "currentPage != NULL").


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@20377 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c40fe37f0b9ed23d499abfbb4dbc51ae282bba1b 30-Aug-2006 Axel Dörfler <axeld@pinc-software.de>

* Reverted the changes r17693 made to vfs_get_vnode_cache(); instead, vm_create_vnode_cache()
will now grab a reference to the vnode as well if successful. This way, vfs_get_vnode_cache()
now actually works how it should: it will now always grab a reference to the cache and
its underlying vnode. This removes an extra reference to the vnode (and vm_cache) that
got ignored before and prevented volumes to be unmounted (or file caches to be removed).
Thanks to Korli for pointing this out.
* file_cache_create() is now aware of that extra vnode reference and releases it; unmounting
volumes is now working again as it should.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@18716 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 517dfdf48bbec4016f03572b1e92de4add00e9ba 22-May-2006 Axel Dörfler <axeld@pinc-software.de>

The cache did not correctly retrieve the file map when the file was fragmented
into more than MAX_FILE_IO_VECS pieces. This could have very weird consequences
like overwriting data outside the file (but on that same partition only).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@17536 a95241bf-73f2-0310-859d-f6bbb57e9c96


# bd5ed534cff597ba657854c6165caf59bdede30b 22-May-2006 Stephan Aßmus <superstippi@gmx.de>

a fix to the file cache (done by Axel), that prevents a crash when
reading the file map of certain files (hope I got that more or less
right)


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@17525 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 97e069713b80839b9dd5441b9f4d37d891b452d8 12-Apr-2006 Axel Dörfler <axeld@pinc-software.de>

Added a "reenter" parameter to the {read|write}_pages() functions to give file
systems a chance to know if they have locked already.
This fixes a locking problem in BFS where one thread tried to acquire two read
locks (where someone else trying to acquire a write lock would have caused a
dead lock).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@17108 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 8645479b368deebb3c4d2ce2d90dbe1d901e72ed 11-Apr-2006 Axel Dörfler <axeld@pinc-software.de>

* cache_io() could insert a second page at the same position in the vm_cache
since it unlocked the cache while waiting on a busy page. Now, we're filling
the pending request before unlocking the cache.
* Fixed the deadlock I mentioned in the last commit: if a page fault happens
at the same time we're trying to read/write from/to a page, we no longer
fight for the BFS inode lock, but eventually doing the job twice if needed.
Will need to go over the "write modified" functions to make sure they are
behaving as well.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@17091 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 8413da2383acc3f815c6493002bee57a05a74aa5 05-Mar-2006 Axel Dörfler <axeld@pinc-software.de>

Like the comment above said, vfs_lookup_vnode() doesn't grab a ref to the vnode,
so we better shouldn't release it.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16591 a95241bf-73f2-0310-859d-f6bbb57e9c96


# b27079971526ef85d166fae1e0fd658455c3d79d 26-Feb-2006 Stephan Aßmus <superstippi@gmx.de>

* read_chunk_into_cache() might not have put back physical pages
in case of a read error.

(coded by axeld)



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16510 a95241bf-73f2-0310-859d-f6bbb57e9c96


# d31d9974ba4c07db6ce7b6702f373dde757bdc96 07-Feb-2006 Axel Dörfler <axeld@pinc-software.de>

Slightly improved debug output.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@16288 a95241bf-73f2-0310-859d-f6bbb57e9c96


# ea2cd27e57272fe016f285c1762252da775ddfc2 21-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

* Fixed the strange vm_cache_ref reference count mechanism: now, a fresh
vm_cache_ref starts with a reference count of 1. When acquiring a vm_cache,
you no longer need to worry if that should go through the vm_store, or not;
as it now always does.
* map_backing_store() no longer needs to play with the vm_cache_ref
references.
* that simplified some code.
* vfs_get_vnode_cache() now grabs a reference to the cache, if successful.
* better balanced vnode ownership on vnode_store creation (vnode_store
released the vnode before if its creation failed).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15641 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 79f73dbc56ac58d027f33ad260c1eefbc1730935 20-Dec-2005 Axel Dörfler <axeld@pinc-software.de>

* vm_page::offset is now called cache_offset and is now an uint32 instead of off_t;
this saves 4 bytes per page. To compensate the loss of bytes, the offset is now
stored in page size units, that's enough to address 2^44 or 16 TB (which is now
the maximal supported file size!).
* Renamed vm_page::ppn to physical_page_number.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15637 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 2b028fcaa02836669081839aa20fdf814081b1c0 23-Nov-2005 Axel Dörfler <axeld@pinc-software.de>

Removed one TODO from the list: in case pages_io() fails, read_chunk_into_cache()
will no longer panic, but free its allocated pages.
I ran into this because BFS managed to create a file without data stream but
with a length larger than 0...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@15093 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 1c88c851adbc9991aa81284493335993d4a99504 19-Oct-2005 Axel Dörfler <axeld@pinc-software.de>

Turned off debug output again - also temporarily removed the file cache warning
about our block_io module not honouring the total length in read_pages().
Removed drops into the debugger when there is a block without an "original"
data buffer - that's completely normal and happens when someone asks for
a cleared block that is not yet in the cache.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14437 a95241bf-73f2-0310-859d-f6bbb57e9c96


# b3373253a4f6c40429fa6305f57261b34001d18a 28-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

The generic syscall now returns more correct status codes.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14078 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 4784eca19deaf9aee8088900b6a579bede9d6452 22-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

cache_io() did not set the state of a page to PAGE_STATE_MODIFIED when it wrote
to it - if that page was read from disk unmodified before (or written back in
the mean time), the updated contents coult not be detected, and therefore, were
never written back.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14048 a95241bf-73f2-0310-859d-f6bbb57e9c96


# e6b682541e3fd31dd54d99b20fd8002f9d132faf 20-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

Fixed a stupid bug in the recently updated write_chunk_to_cache(): lastPageOffset
was incorrectly calculated, and could therefore cause all sorts of troubles with
writes over 4 kB.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14011 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 99e1c71ee6aa9a471306831f2d2a2432eb21cf00 20-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

- fixed a locking bug in the file cache: get_file_map() could been called with either
only a read lock on the inode, or without a lock at all (in case of prefetching) - it
is now using the cache_ref's lock to make sure it's retrieved only once and doesn't
waste/corrupts any memory.
- since that fix made it mandatory to fix write_chunk_into_cache() to not call pages_io()
while holding the cache_ref's lock (was a to-do item before), I changed it to make
that possible.
- It now also supports write-through caches in theory - as there is no way yet to tell
a cache to work this way.
- Optimized for the not so uncommon case of writing the last part of a file that is not
a multiple of the page size - it won't call pages_io() anymore then, but zero the rest
of the page directly.
- vm_page_write_modified() is now calling write_page() without holding the cache_ref's
lock as well.
The updated write_chunk_to_cache() is not so well tested, though, but appears to work
so far.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@14006 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 5913a657dfbf8b0786adb4477b9977b01c764b0e 17-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

Added two possible "ToDo" comments.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13970 a95241bf-73f2-0310-859d-f6bbb57e9c96


# b96ea69c37a666794d56a5018601f93b7747569a 12-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

You really can do the same mistake twice, even in two subsequent lines of code...
get_file_map() now seems to work correctly in all regular cases.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13947 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 139353cf1ad4234e55a5694352bdaacee4d6e432 12-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

Fixed two bugs in the new file extent caching code:
- under certain situations, the wrong extent could be chosen (off-by-one error)
- the resulting length of the first vector was incorrectly calculated for all
extents but the first
Improved error checking a bit.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13941 a95241bf-73f2-0310-859d-f6bbb57e9c96


# eab435cd59a30d144ecc077fbdbc1bb37b3eb5a6 05-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

vfs_get_vnode_cache() now only allocates a new cache if requested: this
prevents the system to allocate caches for files that don't use or have
a file cache (ie. only those can be mmap()ed!).
Therefore, cache_prefetch() no longer crashes when trying to prefetch
files without a file cache.
read_into_cache() no longer does anything if the requested size is 0.
Fixed a bug in cache_prefetch_vnode(): if the cache couldn't be retrieved,
it put the vnode, but didn't own it (the caller does).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13904 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 95839f5c9276596236cd5b6658465e588e2811e0 03-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

The file cache now has another init method that is called after the boot
device becomes available.
Currently, it opens the "launch_speedup" module (if available), later it
should consult a settings file for what to do.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13894 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 11a3346caa5c6a9e6d9ca6d7935f7ae79d892a22 01-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

The file extent map is now cached as well. Code is not optimal yet, and almost not
tested for fragmented files - but it seems to work good enough for now.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13870 a95241bf-73f2-0310-859d-f6bbb57e9c96


# b72cf2601fbc931d2b3224e285eb26d09b20ea5e 01-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

Renamed vfs_vnode_acquire_ref() to vnode_acquire_vnode().
Removed vfs_vnode_release_ref(), as vfs_put_vnode() already does the same thing.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13867 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 186857529ef189e9f3f44b567e07e75a9be38b9a 01-Aug-2005 Axel Dörfler <axeld@pinc-software.de>

cache_prefetch() now comes in two flavours: one with a direct vnode pointer,
the other one with usual device/inode ID pair.
Both versions now accept an offset/size pair to specify the region of the
file to be prefetched - this may be turned into a file_vec_io array later on.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13866 a95241bf-73f2-0310-859d-f6bbb57e9c96


# c9955bf00ae29224a9762f214361b34751ddb535 13-Jul-2005 Axel Dörfler <axeld@pinc-software.de>

Added new but empty function file_cache_invalidate_file_map().
This is needed in case the on-disk representation of a file changes (due to reorganization/defragmentation).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13660 a95241bf-73f2-0310-859d-f6bbb57e9c96


# 2d690920ac4d0cd27eb3c118fb2b0862615869e0 13-Apr-2005 Axel Dörfler <axeld@pinc-software.de>

Renamed system/core to system/kernel.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@12360 a95241bf-73f2-0310-859d-f6bbb57e9c96