However, I put all these documentation in a two pages here.
18. ) Documentation for crash command list:
NAME
list - linked list
SYNOPSIS
list [[-o] offset] [-e end] [-s struct[.member[,member]]] [-H] start
DESCRIPTION
This command dumps the contents of a linked list. The entries in a linked
list are typically data structures that are tied together in one of two
formats:
1. A starting address points to a data structure; that structure contains
a member that is a pointer to the next structure, and so on. The list
typically ends when a "next" pointer value contains one of the
following:
a. a NULL pointer.
b. a pointer to the start address.
c. a pointer to the first item pointed to by the start address.
d. a pointer to its containing structure.
2. Most Linux lists are linked via embedded list_head structures contained
within the data structures in the list. The linked list is headed by an
external LIST_HEAD, which is simply a list_head structure initialized to
point to itself, signifying that the list is empty:
In the case of list_head-type lists, the "next" pointer is the address
of the embedded list_head structure in the next structure, and not the
address of the structure itself. The list typically ends when the
list_head's next pointer points back to the LIST_HEAD address.
This command can handle both types of linked list; in both cases the list
of addresses that are dumped are the addresses of the data structures
themselves.
The arguments are as follows:
[-o] offset The offset within the structure to the "next" pointer
(default is 0). If non-zero, the offset may be entered
in either of two manners:
1. In "structure.member" format; the "-o" is not necessary.
2. A number of bytes; the "-o" is only necessary on processors
where the offset value could be misconstrued as a kernel
virtual address.
-e end If the list ends in a manner unlike the typical manners that
are described above, an explicit ending address value may be
entered.
-s struct For each address in list, format and print as this type of
structure; use the "struct.member" format in order to display
a particular member of the structure. To display multiple
members of a structure, use a comma-separated list of members.
The meaning of the "start" argument, which can be expressed either
symbolically or in hexadecimal format, depends upon whether the -H option
is pre-pended or not:
start The address of the first structure in the list.
-H start The address of the list_head structure, typically expressed
symbolically, but also can be an expression evaluating to the
address of the starting list_head structure.
EXAMPLES
Note that each task_struct is linked to its parent's task_struct via the
p_pptr member:
The list of currently-registered file system types are headed up by a
struct file_system_type pointer named "file_systems", and linked by
the "next" field in each file_system_type structure. The following
sequence displays the structure address followed by the name and
fs_flags members of each registered file system type:
crash> p file_systems
file_systems = $1 = (struct file_system_type *) 0xc03adc90
crash> list file_system_type.next -s file_system_type.name,fs_flags 0xc03adc90
c03adc90
name = 0xc02c05c8 "rootfs",
fs_flags = 0x30,
c03abf94
name = 0xc02c0319 "bdev",
fs_flags = 0x10,
c03acb40
name = 0xc02c07c4 "proc",
fs_flags = 0x8,
c03e9834
name = 0xc02cfc83 "sockfs",
fs_flags = 0x10,
c03ab8e4
name = 0xc02bf512 "tmpfs",
fs_flags = 0x20,
c03ab8c8
name = 0xc02c3d6b "shm",
fs_flags = 0x20,
c03ac394
name = 0xc02c03cf "pipefs",
fs_flags = 0x10,
c03ada74
name = 0xc02c0e6b "ext2",
fs_flags = 0x1,
c03adc74
name = 0xc02c0e70 "ramfs",
fs_flags = 0x20,
c03ade74
name = 0xc02c0e76 "hugetlbfs",
fs_flags = 0x20,
c03adf8c
name = 0xc02c0f84 "iso9660",
fs_flags = 0x1,
c03aec14
name = 0xc02c0ffd "devpts",
fs_flags = 0x8,
c03e93f4
name = 0xc02cf1b9 "pcihpfs",
fs_flags = 0x28,
e0831a14
name = 0xe082f89f "ext3",
fs_flags = 0x1,
e0846af4
name = 0xe0841ac6 "usbdevfs",
fs_flags = 0x8,
e0846b10
name = 0xe0841acf "usbfs",
fs_flags = 0x8,
e0992370
name = 0xe099176c "autofs",
fs_flags = 0x0,
e2dcc030
name = 0xe2dc8849 "nfs",
fs_flags = 0x48000,
In some kernels, the system run queue is a linked list headed up by the
"runqueue_head", which is defined like so:
static LIST_HEAD(runqueue_head);
The run queue linking is done with the "run_list" member of the task_struct:
Therefore, to view the list of task_struct addresses in the run queue,
either of the following commands will work:
crash> list task_struct.run_list -H runqueue_head
f79ac000
f7254000
f7004000
crash> list 60 -H runqueue_head
f79ac000
f7254000
f7004000
Lastly, in some kernel versions, the vfsmount structures of the mounted
filesystems are linked by the LIST_HEAD "vfsmntlist", which uses the
mnt_list list_head of each vfsmount structure in the list. To dump each
vfsmount structure in the list, append the -s option:
DESCRIPTION
This command dumps the kernel log_buf contents in chronological order.
-m Display the message log level preceding each message.
EXAMPLES
Dump the kernel message buffer:
crash> log
Linux version 2.2.5-15smp (root@mclinux1) (gcc version egcs-2.91.66 19990
314/Linux (egcs-1.1.2 release)) #1 SMP Thu Aug 26 11:04:37 EDT 1999
Intel MultiProcessor Specification v1.4
Virtual Wire compatibility mode.
OEM ID: DELL Product ID: WS 410 APIC at: 0xFEE00000
Processor #0 Pentium(tm) Pro APIC version 17
Processor #1 Pentium(tm) Pro APIC version 17
I/O APIC #2 Version 17 at 0xFEC00000.
Processors: 2
mapped APIC to ffffe000 (fee00000)
mapped IOAPIC to ffffd000 (fec00000)
Detected 447696347 Hz processor.
Console: colour VGA+ 80x25
Calibrating delay loop... 445.64 BogoMIPS
...
8K byte-wide RAM 5:3 Rx:Tx split, autoselect/Autonegotiate interface.
MII transceiver found at address 24, status 782d.
Enabling bus-master transmits and whole-frame receives.
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
nfsd_init: initialized fhcache, entries=256
...
Do the same thing, but also show the log level preceding each message:
crash> log -m
<4>Linux version 2.2.5-15smp (root@mclinux1) (gcc version egcs-2.91.66 19990 314/Linux (egcs-1.1.2 release)) #1 SMP Thu Aug 26 11:04:37 EDT 1999 <4>Intel MultiProcessor Specification v1.4 <4> Virtual Wire compatibility mode. <4>OEM ID: DELL Product ID: WS 410 APIC at: 0xFEE00000 <4>Processor #0 Pentium(tm) Pro APIC version 17 <4>Processor #1 Pentium(tm) Pro APIC version 17 <4>I/O APIC #2 Version 17 at 0xFEC00000. <4>Processors: 2 <4>mapped APIC to ffffe000 (fee00000) <4>mapped IOAPIC to ffffd000 (fec00000) <4>Detected 447696347 Hz processor. <4>Console: colour VGA+ 80x25 <4>Calibrating delay loop... 445.64 BogoMIPS ... <6> 8K byte-wide RAM 5:3 Rx:Tx split, autoselect/Autonegotiate interface. <6> MII transceiver found at address 24, status 782d. <6> Enabling bus-master transmits and whole-frame receives. <6>Installing knfsd (copyright (C) 1996 okir@monad.swb.de). <7>nfsd_init: initialized fhcache, entries=256 ...
20. ) Documentation for crash command mach:
NAME mach - machine specific data
SYNOPSIS mach [-cm]
DESCRIPTION This command displays data specific to a machine type.
-c Display each cpu's cpuinfo structure (x86, x86_64 and ia64 only). Display each cpu's x8664_pda structure (x86_64 only), Display the hwrpb_struct, and each cpu's percpu_struct (alpha only). -m Display the physical memory map (x86, x86_64 and ia64 only).
DESCRIPTION With no arguments, this command displays basic information of the currently installed modules, consisting of the module address, name, size, the object file name (if known), and whether the module was compiled with CONFIG_KALLSYMS.
The arguments are concerned with with the loading or deleting of symbolic and debugging data from a module's object file. A modules's object file always contains symbolic data (symbol names and addresses), but contains debugging data only if the module was compiled with the -g CFLAG. In addition, the module may have compiled with CONFIG_KALLSYMS, which means that the module's symbolic data will have been loaded into the kernel's address space when it was installed. If the module was not compiled with CONFIG_KALLSYMS, then only the module's exported symbols will be loaded into the kernel's address space. Therefore, for the purpose of this command, it should noted that a kernel module may have been compiled in one of following manners:
1. If the module was built without CONFIG_KALLSYMS and without the -g CFLAG, then the loading of the module's additional non-exported symbols can be accomplished with this command. 2. If the module was built with CONFIG_KALLSYMS, but without the -g CFLAG, then there is no benefit in loading the symbols from the module object file, because all of the module's symbols will have been loaded into the kernel's address space when it was installed. 3. If the module was built with CONFIG_KALLSYMS and with the the -g CFLAG, then the loading of the module's debugging data can be accomplished with this command. 4. If the module was built without CONFIG_KALLSYMS but with the -g CFLAG, then the loading of the both module's symbolic and debugging data can be accomplished with this command.
-s module [objfile] Loads symbolic and debugging data from the object file for the module specified. If no objfile argument is appended, a search will be made for an object file consisting of the module name with a .o or .ko suffix, starting at the /lib/modules/ directory on the host system. If an objfile argument is appended, then that file will be used. -d module Deletes the symbolic and debugging data of the module specified. -S [directory] Load symbolic and debugging data from the object file for all loaded modules. For each module, a search will be made for an object file consisting of the module name with a .o or.ko suffix, starting at the /lib/modules/ directory of the host system. If a directory argument is appended, then the search will be restricted to that directory. -D Deletes the symbolic and debugging data of all modules. -r Passes the -readnow flag to the embedded gdb module, which will override the two-stage strategy that it uses for reading symbol tables from module object files. -R Reinitialize module data. All currently-loaded symbolic and debugging data will be deleted, and the installed module list will be updated (live system only). -g When used with -s or -S, add a module object's section start and end addresses to its symbol list. -o Load module symbols with old mechanism.
After symbolic and debugging data have been loaded, backtraces and text disassembly will be displayed appropriately. Depending upon the processor architecture, data may also printed symbolically with the "p" command; at a minimum, the "rd" command may be used with module data symbols.
If crash can recognize that the set of modules has changed while running a session on a live kernel, the module data will be reinitialized the next time this command is run; the -r option forces the reinitialization.
SYNOPSIS mount [-f] [-i] [-n pid|task] [vfsmount|superblock|devname|dirname|inode]
DESCRIPTION This command displays basic information about the currently-mounted filesystems. The per-filesystem dirty inode list or list of open files for the filesystem may also be displayed.
-f dump dentries and inodes for open files in each filesystem. -i dump all dirty inodes associated with each filesystem; only supported on kernels with super_block.s_dirty linked list.
For kernels supporting namespaces, the -n option may be used to display the mounted filesystems with respect to the namespace of a specified task:
-n pid a process PID. -n task a hexadecimal task_struct pointer.
Specific filesystems may be selected using the following forms:
vfsmount hexadecimal address of filesystem vfsmount structure. superblock hexadecimal address of filesystem super_block structure. devname device name of filesystem. dirname directory where filesystem is mounted. inode hexadecimal address of an open inode of a filesystem.
-a display the ARP cache. -s display open network socket/sock addresses, their family and type, and for INET and INET6 families, their source and destination addresses and ports. -S displays open network socket/sock addresses followed by a dump of both structures. -n addr translates an IPv4 address expressed as a decimal or hexadecimal value into a standard numbers-and-dots notation. -R ref socket or sock address, or file descriptor. pid a process PID. taskp a hexadecimal task_struct pointer.
If no arguments are entered, the list of network devices, names and IP addresses are displayed. The -R option, typically invoked from "foreach net", and in conjunction with the -s or -S options, searches for references to a socket address, sock address, or a file descriptor; if found, only the referenced fd/socket/sock data will be displayed.
EXAMPLES Display the network device list:
crash> net DEVICE NAME IP ADDRESS(ES) c0249f20 lo 127.0.0.1 c7fe6d80 eth0 10.1.8.20
Dump the ARP cache:
crash> net -a IP ADDRESS HW TYPE HW ADDRESS DEVICE STATE 0.0.0.0 UNKNOWN 00 00 00 00 00 00 lo 40 (NOARP) 192.168.1.1 ETHER 00:50:54:fe:ef:23 eth0 04 (STALE) 192.168.1.10 ETHER 00:90:27:9c:6c:79 eth0 02 (REACHABLE) 192.168.1.118 ETHER 00:c0:4f:60:00:e2 eth0 02 (REACHABLE)
Display the sockets for PID 2517, using both -s and -S output formats:
DESCRIPTION This command passes its arguments on to gdb "print" command for evaluation.
expression The expression to be evaluated. -x override default output format with hexadecimal format. -d override default output format with decimal format. -u the expression evaluates to a user address reference.
The default output format is decimal, but that can be changed at any time with the two built-in aliases "hex" and "dec". Alternatively, there are two other built-in aliases, "px" and "pd", which force the command output to be displayed in hexadecimal or decimal, without changing the default mode.
However, I put all these documentation in a two pages here.
# cat
/tmp/crash-help-strings.txt
*filesmodrunqunion
aliasforeachmountsearchvm
asciifusernetsetvtop
btgdbpsigwaitq
btophelppsstructwhatis
devirqpteswapwr
diskmemptobsymq
evallistptovsys
exitlogrdtask
extendmachrepeattimer
#
1. ) Documentation for crash command *:
NAME
* - pointer-to short-cut
SYNOPSIS
* (struct or union command arguments)
DESCRIPTION
This command is a short-cut command that replaces the requirement to enter
"struct" or "union" command names. For details on the arguments to
those commands, enter "help struct" or "help union".
EXAMPLES
Dump the page structure at address c02943c0:
DESCRIPTION
This command creates an alias for a given command string. If no arguments
are entered, the current list of aliases are displayed. If one argument is
entered, the command string for that alias, if any, is displayed.
alias the single word to be used as an alias
command string the word(s) that will be substituted for the alias
Aliases may be created in four manners:
1. entering the alias in $HOME/.crashrc.
2. entering the alias in .crashrc in the current directory.
3. executing an input file containing the alias command.
4. during runtime with this command.
During initialization, $HOME/.crashrc is read first, followed by the
.crashrc file in the current directory. Aliases in the .crashrc file
in the current directory override those in $HOME/.crashrc. Aliases
entered with this command or by runtime input file override those
defined in either .crashrc file. Aliases may be deleted by entering an
empty string for the second argument. If redirection characters are to
be part of the command string, the command string must be enclosed by
quotation marks.
Note that there are a number of helpful built-in aliases -- see the
first example below.
EXAMPLES
Display the currently-defined aliases, which in this example, only
consist of the built-in aliases:
crash> alias
ORIGIN ALIAS COMMAND
builtin man help
builtin ? help
builtin quit q
builtin sf set scroll off
builtin sn set scroll on
builtin hex set radix 16
builtin dec set radix 10
builtin g gdb
builtin px p -x
builtin pd p -d
builtin for foreach
builtin size *
builtin dmesg log
builtin lsmod mod
builtin last ps -l
Create a new alias to be added to the list:
crash> alias kp kmem -p
ORIGIN ALIAS COMMAND
runtime kp kmem -p
0 1 2 3 4 5 6 7
+-------------------------------
0 | NUL DLE SP 0 @ P ' p
1 | SOH DC1 ! 1 A Q a q
2 | STX DC2 " 2 B R b r
3 | ETX DC3 # 3 C S c s
4 | EOT DC4 $ 4 D T d t
5 | ENQ NAK % 5 E U e u
6 | ACK SYN & 6 F V f v
7 | BEL ETB ` 7 G W g w
8 | BS CAN ( 8 H X h x
9 | HT EM ) 9 I Y i y
A | LF SUB * : J Z j z
B | VT ESC + ; K [ k {
C | FF FS , < L \ l |
D | CR GS _ = M ] m }
E | SO RS . > N ^ n ~
F | SI US / ? O - o DEL
DESCRIPTION
Display a kernel stack backtrace. If no arguments are given, the stack
trace of the current context will be displayed.
-a displays the stack traces of the active task on each CPU.
(only applicable to crash dumps)
-g displays the stack traces of all threads in the thread group of
the target task; the thread group leader will be displayed first.
-r display raw stack data, consisting of a memory dump of the two
pages of memory containing the task_union structure.
-t display all text symbols found from the last known stack location
to the top of the stack. (helpful if the back trace fails)
-T display all text symbols found from just above the task_struct or
thread_info to the top of the stack. (helpful if the back trace
fails or the -t option starts too high in the process stack).
-l show file and line number of each stack trace text location.
-e search the stack for possible kernel and user mode exception frames.
-E search the IRQ stacks (x86, x86_64 and ppc64), and the exception
stacks (x86_64) for possible exception frames; all other arguments
will be ignored since this is not a context-sensitive operation.
-f display all stack data contained in a frame; this option can be
used to determine the arguments passed to each function; on ia64,
the argument register contents are dumped.
-F similar to -f, except that the stack data is displayed symbolically
when appropriate; if the stack data references a slab cache object,
the name of the slab cache will be displayed in brackets; on ia64,
the substitution is done to the argument register contents.
-o x86: use old backtrace method, permissable only on kernels that were
compiled without the -fomit-frame_pointer.
x86_64: use old backtrace method, which dumps potentially stale
kernel text return addresses found on the stack.
-O x86: use old backtrace method by default, permissable only on kernels
that were compiled without the -fomit-frame_pointer; subsequent usage
of this option toggles the backtrace method.
x86_64: use old backtrace method by default; subsequent usage of this
option toggles the backtrace method.
-R ref display stack trace only if there is a reference to this symbol
or text address.
-I ip use ip as the starting text location.
-S sp use sp as the starting stack frame address.
pid displays the stack trace(s) of this pid.
taskp displays the stack trace the the task referenced by this hexadecimal
task_struct pointer.
Multiple pid and taskp arguments may be specified.
Note that all examples below are for x86 only. The output format will differ
for other architectures. x86 backtraces from kernels that were compiled
with the --fomit-frame-pointer CFLAG occasionally will drop stack frames,
or display a stale frame reference. When in doubt as to the accuracy of a
backtrace, the -t or -T options may help fill in the blanks.
EXAMPLES
Display the stack trace of the active task(s) when the kernel panicked:
crash> bt -a
PID: 286 TASK: c0b3a000 CPU: 0 COMMAND: "in.rlogind"
#0 [c0b3be90] crash_save_current_state at c011aed0
#1 [c0b3bea4] panic at c011367c
#2 [c0b3bee8] tulip_interrupt at c01bc820
#3 [c0b3bf08] handle_IRQ_event at c010a551
#4 [c0b3bf2c] do_8259A_IRQ at c010a319
#5 [c0b3bf3c] do_IRQ at c010a653
#6 [c0b3bfbc] ret_from_intr at c0109634
EAX: 00000000 EBX: c0e68280 ECX: 00000000 EDX: 00000004 EBP: c0b3bfbc
DS: 0018 ESI: 00000004 ES: 0018 EDI: c0e68284
CS: 0010 EIP: c012f803 ERR: ffffff09 EFLAGS: 00000246
#7 [c0b3bfbc] sys_select at c012f803
#8 [c0b3bfc0] system_call at c0109598
EAX: 0000008e EBX: 00000004 ECX: bfffc9a0 EDX: 00000000
DS: 002b ESI: bfffc8a0 ES: 002b EDI: 00000000
SS: 002b ESP: bfffc82c EBP: bfffd224
CS: 0023 EIP: 400d032e ERR: 0000008e EFLAGS: 00000246
Display the stack traces of task f2814000 and PID 1592:
crash> bt f2814000 1592
PID: 1018 TASK: f2814000 CPU: 1 COMMAND: "java"
#0 [f2815db4] schedule at c011af85
#1 [f2815de4] __down at c010600f
#2 [f2815e14] __down_failed at c01061b3
#3 [f2815e24] stext_lock (via drain_cpu_caches) at c025fa55
#4 [f2815ec8] kmem_cache_shrink_nr at c013a53e
#5 [f2815ed8] do_try_to_free_pages at c013f402
#6 [f2815f04] try_to_free_pages at c013f8d2
#7 [f2815f1c] _wrapped_alloc_pages at c01406bd
#8 [f2815f40] __alloc_pages at c014079d
#9 [f2815f60] __get_free_pages at c014083e
#10 [f2815f68] do_fork at c011cebb
#11 [f2815fa4] sys_clone at c0105ceb
#12 [f2815fc0] system_call at c010740c
EAX: 00000078 EBX: 00000f21 ECX: bc1ffbd8 EDX: bc1ffbe0
DS: 002b ESI: 00000000 ES: 002b EDI: bc1ffd04
SS: 002b ESP: 0807316c EBP: 080731bc
CS: 0023 EIP: 4012881e ERR: 00000078 EFLAGS: 00000296
In order to examine each stack frame's contents use the bt -f option.
From the extra frame data that is displayed, the arguments passed to each
function can be determined. Re-examining the PID 1592 trace above:
Typically the arguments passed to a function will be the last values
that were pushed onto the stack by the next higher-numbered function, i.e.,
the lowest stack addresses in the frame above the called function's
stack frame. That can be verified by disassembling the calling function.
For example, the arguments passed from sys_read() to pipe_read() above
are the file pointer, the user buffer address, the count, and a pointer
to the file structure's f_pos field. Looking at the frame #3 data for
sys_read(), the last four items pushed onto the stack (lowest addresses)
are f27ae680, bffed4a3, 00000001, and f27ae6a0 -- which are the 4 arguments
above, in that order. Note that the first (highest address) stack content
in frame #2 data for pipe_read() is c0148be8, which is the return address
back to sys_read().
Dump the text symbols found in the current context's stack:
crash> bt -t
PID: 1357 TASK: c1aa0000 CPU: 0 COMMAND: "lockd"
START: schedule at c01190e0
[c1aa1f28] dput at c0157dbc
[c1aa1f4c] schedule_timeout at c0124cd4
[c1aa1f78] svc_recv at cb22c4d8 [sunrpc]
[c1aa1f98] put_files_struct at c011eb21
[c1aa1fcc] nlmclnt_proc at cb237bef [lockd]
[c1aa1ff0] kernel_thread at c0105826
[c1aa1ff8] nlmclnt_proc at cb237a60 [lockd]
Search the current stack for possible exception frames:
Display the back trace from a dumpfile that resulted from an attempt to
insmod the sample "crash.c" kernel module that comes as part of the
Red Hat netdump package:
DESCRIPTION
This command translates a hexadecimal address to its page number.
EXAMPLES
crash> btop 512a000
512a000: 512a
6. ) Documentation for crash command dev:
NAME
dev - device data
SYNOPSIS
dev [-i | -p | -d]
DESCRIPTION
If no argument is entered, this command dumps character and block
device data.
-i display I/O port usage; on 2.4 kernels, also display I/O memory usage.
-p display PCI device data.
-d display disk I/O statistics:
TOTAL: total number of allocated in-progress I/O requests
SYNC: I/O requests that are synchronous
ASYNC: I/O requests that are asynchronous
READ: I/O requests that are reads (older kernels)
WRITE: I/O requests that are writes (older kernels)
DRV: I/O requests that are in-flight in the device driver
SYNOPSIS
dis [-rludx][-b [num]] [address | symbol | (expression)] [count]
DESCRIPTION
This command disassembles source code instructions starting (or ending) at
a text address that may be expressed by value, symbol or expression:
-r (reverse) displays all instructions from the start of the
routine up to and including the designated address.
-l displays source code line number data in addition to the
disassembly output.
-u address is a user virtual address in the current context;
otherwise the address is assumed to be a kernel virtual address.
If this option is used, then -r and -l are ignored.
-x override default output format with hexadecimal format.
-d override default output format with decimal format.
-b [num] modify the pre-calculated number of encoded bytes to skip after
a kernel BUG ("ud2a") instruction; with no argument, displays
the current number of bytes being skipped. (x86 and x86_64 only)
address starting hexadecimal text address.
symbol symbol of starting text address. On ppc64, the symbol
preceded by '.' is used.
(expression) expression evaluating to a starting text address.
count the number of instructions to be disassembled (default is 1).
If no count argument is entered, and the starting address
is entered as a text symbol, then the whole routine will be
disassembled. The count argument is ignored when used with
the -r option.
EXAMPLES
Disassemble the sys_signal() routine without, and then with, line numbers:
DESCRIPTION This command evaluates an expression or numeric value, and displays its result in hexadecimal, decimal, octal and binary. If the resultant value is an integral number of gigabytes, megabytes, or kilobytes, a short-hand translation of the number will also be shown next to the hexadecimal value. If the most significant bit is set, the decimal display will show both unsigned and signed (negative) values. Expressions must of the format (x operator y), where "x" and "y" may be either numeric values or symbols. The list of operators are:
+ - & | ^ * % / << >>
Enclosing the expression within parentheses is optional except when the "|", "<<" or ">>" operators are used. The single "value" argument may be a number or symbol. Number arguments must be hexadecimal or decimal. A leading "0x" identifies a number as hexadecimal, but is not required when obvious. Numbers may be followed by the letters "k" or "K", "m" or "M", and "g" or "G", which multiplies the value by a factor of 1024, 1 megabyte or 1 gigabyte, respectively. Numeric arguments may be preceded by the one's complement operator ~.
-b Indicate which bit positions in the resultant value are set. -l Numeric arguments are presumed to be 64-bit values, and the result will be expressed as a 64-bit value. (ignored on 64-bit processors) However, if either operand or the resultant value are 64-bit values, then the result will be also be expressed as a 64-bit value.
The -b and -l options must precede the expression or value arguments.
DESCRIPTION This command dynamically loads or unloads crash extension shared object libraries:
shared-object load the specified shared object file; more than one one object file may be entered. -u shared-object unload the specified shared object file; if no file arguments are specified, unload all objects.
If the shared-object filename is not expressed with a fully-qualified pathname, the following directories will be searched in the order shown, and the first instance of the file that is found will be selected:
1. the current working directory 2. the directory specified in the CRASH_EXTENSIONS environment variable 3. /usr/lib64/crash/extensions (64-bit architectures) 4. /usr/lib/crash/extensions
If no arguments are entered, the current set of shared object files and a list of their commands will be displayed. The registered commands contained in each shared object file will appear automatically in the "help" command screen.
An example of a shared object prototype file, and how to compile it into a shared object, is appended below.
CREATING A SHARED OBJECT The extend command loads shared object files using dlopen(3), which in turn calls the shared object's _init() function. The shared object's _init() function should register its command set by calling register_extension(), passing it a pointer to an array of one or more structures of the following type:
Each command_table_entry structure contains the ASCII name of a command, the command's function address, a pointer to an array of help data strings, and a flags field. The help_data field is optional; if it is non-NULL, it should point to an array of character strings used by the "help" command, and during command failures. The flags field currently has one available bit setting, REFRESH_TASK_TABLE, which should be set if it is preferable to reload the current set of running processes just prior to executing the command (on a live system). Terminate the array of command_table_entry structures with an entry with a NULL command name.
Below is an example shared object file consisting of just one command, called "echo", which simply echoes back all arguments passed to it. Note the comments contained within it for further details. Cut and paste the following output into a file, and call it, for example, "echo.c". Then compiled in either of two manners. Either manually like so:
where must be one of the MACHINE_TYPE #define's in defs.h, and where $(TARGET_CFLAGS) is the same as it is declared in the top-level Makefile after a build is completed. Or alternatively, the "echo.c" file can be copied into the "extensions" subdirectory, and compiled automatically like so:
make extensions
The echo.so file may be dynamically linked into crash during runtime, or during initialization by putting "extend echo.so" into a .crashrc file located in the current directory, or in the user's $HOME directory.
---------------------------------- cut here ----------------------------------
#include "defs.h" /* From the crash source top-level directory */
void cmd_echo(); /* Declare the commands and their help data. */ char *help_echo[];
static struct command_table_entry command_table[] = { "echo", cmd_echo, help_echo, 0, /* One or more commands, */ NULL, /* terminated by NULL, */ };
_init() /* Register the command set. */ { register_extension(command_table); }
/* * The _fini() function is called if the shared object is unloaded. * If desired, perform any cleanups here. */ _fini() { }
/* * Arguments are passed to the command functions in the global args[argcnt] * array. See getopt(3) for info on dash arguments. Check out defs.h and * other crash commands for usage of the myriad of utility routines available * to accomplish what your task. */ void cmd_echo() { int c;
while (args[optind]) fprintf(fp, "%s ", args[optind++]);
fprintf(fp, "\n"); }
/* * The optional help data is simply an array of strings in a defined format. * For example, the "help echo" command will use the help_echo[] string * array below to create a help page that looks like this: * * NAME * echo - echoes back its arguments * * SYNOPSIS * echo arg ... * * DESCRIPTION * This command simply echoes back its arguments. * * EXAMPLE * Echo back all command arguments: * * crash> echo hello, world * hello, world * */
char *help_echo[] = { "echo", /* command name */ "echo", /* command name */ "echoes back its arguments", /* short description */ "arg ...", /* argument synopsis, or " " if none */
" This command simply echoes back its arguments.", "\nEXAMPLE", " Echo back all command arguments:\n", " crash> echo hello, world", " hello, world", NULL }; 11. ) Documentation for crash command files:
DESCRIPTION This command displays information about open files of a context. It prints the context's current root directory and current working directory, and then for each open file descriptor it prints a pointer to its file struct, a pointer to its dentry struct, a pointer to the inode, the file type, and the pathname. If no arguments are entered, the current context is used. The -R option, typically invoked from "foreach files", searches for references to a supplied number, address, or filename argument, and prints only the essential information leading up to and including the reference. The -d option is not context specific, and only shows the data requested.
-d dentry given a hexadecimal dentry address, display its inode, super block, file type, and full pathname. -R reference search for references to this file descriptor number, filename, or dentry, inode, or file structure address. pid a process PID. taskp a hexadecimal task_struct pointer.
EXAMPLES Display the open files of the current context:
DESCRIPTION This command allows for a an examination of various kernel data associated with any, or all, tasks in the system, without having to set the context to each targeted task.
pid perform the command(s) on this PID. taskp perform the command(s) on task referenced by this hexadecimal task_struct pointer. name perform the command(s) on all commands with this name. If the command name can be confused with a foreach command name, then precede the name string with a "\". user perform the command(s) on all user (non-kernel) threads. kernel perform the command(s) on all kernel threads. active perform the command(s) on the active thread on each CPU.
If none of the task-identifying arguments above are entered, the command will be performed on all tasks.
command select one or more of the following commands to be run on the tasks selected, or on all tasks:
bt run the "bt" command (optional flags: -r -t -l -e -R -f -F -o) vm run the "vm" command (optional flags: -p -v -m -R) task run the "task" command (optional flags: -R -d -x) files run the "files" command (optional flag: -R) net run the "net" command (optional flags: -s -S -R) set run the "set" command sig run the "sig" command (optional flag: -g) vtop run the "vtop" command (optional flags: -c -u -k)
flag Pass this optional flag to the command selected. argument Pass this argument to the command selected.
A header containing the PID, task address, cpu and command name will be pre-pended before the command output for each selected task. Consult the help page of each of the command types above for details.
Display the task_struct structure for each "bash" command:
crash> foreach bash task ...
Display the open files for all tasks:
crash> foreach files ...
13. ) Documentation for crash command fuser:
NAME fuser - file users
SYNOPSIS fuser [pathname | inode]
DESCRIPTION This command displays the tasks using specified files or sockets. Tasks will be listed that reference the file as the current working directory, root directory, an open file descriptor, or that mmap the file. If the file is held open in the kernel by the lockd server on behalf of a client discretionary file lock, the client hostname is listed.
pathname the full pathname of the file. inode the hexadecimal inode address for the file.
EXAMPLES Display the tasks using file /usr/lib/libkfm.so.2.0.0
DESCRIPTION This command passes its arguments directly to gdb for processing. This is typically not necessary, but where ambiguities between crash and gdb command names exist, this will force the command to be executed by gdb.
EXAMPLES crash> gdb help List of classes of commands:
aliases -- Aliases of other commands breakpoints -- Making program stop at certain points data -- Examining data files -- Specifying and examining files internals -- Maintenance commands obscure -- Obscure features running -- Running the program stack -- Examining the stack status -- Status inquiries support -- Support facilities tracepoints -- Tracing of program execution without stopping the program user-defined -- User-defined commands
Type "help" followed by a class name for a list of commands in that class. Type "help" followed by command name for full documentation. Command name abbreviations are allowed if unambiguous.
