From: Andi Kleen Avoid deadlock when kernel fault happens inside mmap sem. Signed-off-by: Andi Kleen Signed-off-by: Andrew Morton --- 25-akpm/arch/x86_64/mm/fault.c | 22 +++++++++++++++++++++- 1 files changed, 21 insertions(+), 1 deletion(-) diff -puN arch/x86_64/mm/fault.c~x86-64-avoid-deadlock-in-page-fault-handler arch/x86_64/mm/fault.c --- 25/arch/x86_64/mm/fault.c~x86-64-avoid-deadlock-in-page-fault-handler Tue Sep 14 17:02:30 2004 +++ 25-akpm/arch/x86_64/mm/fault.c Tue Sep 14 17:02:30 2004 @@ -315,7 +315,27 @@ asmlinkage void do_page_fault(struct pt_ goto bad_area_nosemaphore; again: - down_read(&mm->mmap_sem); + /* When running in the kernel we expect faults to occur only to + * addresses in user space. All other faults represent errors in the + * kernel and should generate an OOPS. Unfortunatly, in the case of an + * erroneous fault occuring in a code path which already holds mmap_sem + * we will deadlock attempting to validate the fault against the + * address space. Luckily the kernel only validly references user + * space from well defined areas of code, which are listed in the + * exceptions table. + * + * As the vast majority of faults will be valid we will only perform + * the source reference check when there is a possibilty of a deadlock. + * Attempt to lock the address space, if we cannot we then validate the + * source. If this is invalid we can skip the address space check, + * thus avoiding the deadlock. + */ + if (!down_read_trylock(&mm->mmap_sem)) { + if ((error_code & 4) == 0 && + !search_exception_tables(regs->rip)) + goto bad_area_nosemaphore; + down_read(&mm->mmap_sem); + } vma = find_vma(mm, address); if (!vma) _