How to see exactly which command in a Linux system is using most memory or CPU

Following command will return the commands/processes on the linux box and its %memory and cpu consumption.
top c

How to manage resource on Linux machines using ulimit

What is ulimit

Source of this article: http://www.linuxhowtos.org/Tips%20and%20Tricks/ulimit.htm

The ulimit  allow to limit system-wide resource use. This can help a lot in system administration, e.g. when a user starts too many processes and therefore makes the system unresponsive for other users.

Code Listing 1: ulimit example

# ulimit -a 
core file size          (blocks, -c) 0 
data seg size           (kbytes, -d) unlimited 
file size               (blocks, -f) unlimited 
pending signals                 (-i) 8191 
max locked memory       (kbytes, -l) 32 
max memory size         (kbytes, -m) unlimited 
open files                      (-n) 1024 
pipe size            (512 bytes, -p) 8 
POSIX message queues     (bytes, -q) 819200 
stack size              (kbytes, -s) 8192 
cpu time               (seconds, -t) unlimited 
max user processes              (-u) 8191 
virtual memory          (kbytes, -v) unlimited 
file locks                      (-x) unlimited 

All these settings can be manipulated. A good example is this bash forkbomb that forks as many processes as possible and can crash systems where no user limits are set:

Warning: Do not run this in a shell! If no limits are set your system will either become unresponsive or might even crash.

Code Listing 2: A bash forkbomb

$ :(){ :|:& };:

Now this is not good - any user with shell access to your box could take it down. But if that user can only start 30 processes the damage will be minimal. So let's set a process limit:

Gentoo Note: A too small number of processes can break the use of portage. So, don't be too strict.

Code Listing 3: Setting a process limit

# ulimit -u 30 
# ulimit -a 
… 
max user processes              (-u) 30 
…

If you try to run the forkbomb now it should run, but throw error messages "fork: resource temporarily unavailable". This means that your system has not allowed the forkbomb to start more processes. The other options of ulimit can help with similar problems, but you should be careful that you don't lock yourself out - setting data seg size too small will even prevent bash from starting.

More details about ulimit

Source of this article: http://answers.google.com/answers/threadview?id=311442

CPU TIME - Limits the number of CPU seconds a process can use (not
clock seconds).  CPU time is the amount of time the CPU actual spends
executing processor instructions and is often much less than the total
program "runs time".  A program that did math for 10 seconds would use
a lot of CPU time, where as a program that simply paused for 10
seconds before display a message would use next to none.

When a process exceeds the soft CPU time limit it will be sent a XCPU
signal (and terminated if the signal isn't caught).  When it exceeds
the hard limit it will be sent a kill signal.

VIRTUAL MEMORY SIZE - the most useful of the memory-related
limitations, because it includes all types of memory, including the
stack, the heap, and memory-mapped files. Attempts to allocate memory
in excess of this limit will fail with an out-of-memory error.
Specified in kilobytes, in the bash shell, as found in Linux and many
other systems.

DATA SEGMENT SIZE - Limits the amount of memory that a process can
allocate on the heap, as with malloc, calloc, C++ "new," and most
object creation in higher-level languages. Specified in kilobytes, in
the bash shell, as found in Linux and many other systems.

STACK SIZE - Limits the amount of memory a process can allocate on the
stack, as in the case of local variables in C, C++ and many other
languages. Limiting the stack size stops runaway recursive function
calls; however, it is possible to legitimately allocate large arrays
of data on the stack without unreasonable recursion. Specified in
kilobytes, in the bash shell, as found in Linux and many other
systems.

FILE SIZE - Limits the maximum size of any one file a process can
create. Specified in 512-byte blocks, in the bash shell, as found in
Linux and many other systems.

MAX USER PROCESSES - limits the number of processes the current user
is permitted to have in the process table at one time. Attempts to
start additional processes will fail.

OPEN FILES - limits the number of file descriptors belonging to a
single process. "File descriptors" includes not only files but also
sockets for Internet communication. Attempts to open additional file
descriptors will fail.

CORE FILE SIZE - Limits the size of a "core" file left behind when a
process encounters a segmentation fault or other unexpected fatal
error. Core files are images of the entire memory space used by the
process at the time of the crash, and can be used to examine the state
of the process at that time with a debugger, such as gdb. In most
cases core files are of limited utility if the process was not
compiled with debugging flags enabled. Disabling core files with a
core file size limit of zero is reasonable when the processes expected
are not C/C++ applications compiled with -g or similar or no such
debugging capability is desired. Specified in 512-byte blocks.

LOCKED MEMORY - this parameter limits the maximum amount of memory
that can be "locked down" to a specific address in physical memory by
a given process. In practice, only the root user can lock memory in
this fashion. Specified in kilobytes, in the bash shell, as found in
Linux and many other systems.

MAX RESIDENT SET SIZE - this parameter limits the amount of memory
that can be "swapped in" to physical RAM on behalf of any one process.
While patches have been offered, this limit is not actually enforced
in mainstream Linux kernels; it briefly appeared in the 2.6 series but
has been removed for now due to problems. Specified in kilobytes, in
the bash shell, as found in Unix and many other systems.

MAX TOTAL SOCKET BUFFER SIZE - this parameter limits the total amount
of memory that may be taken up by buffers holding network data on
behalf of a given process. Originated as a response to
denial-of-service attacks which worked by initiating large numbers of
connections and flooding them with input; the operating system would
buffer this data patiently while waiting for the receiving process to
cope with it. Implemented in FreeBSD; not implemented in Linux, which
does not address this issue via a configurable limit. Specified in
bytes.

PIPE SIZE - when two Unix processes communicate via a pipe or FIFO
(first in first out) buffer, as in the simple case of paging through a
directory listing with the command "ls | more", the output of the
first command is buffered before transmission to the second. The size
of this buffer, in bytes, is the pipe size. This is typically not an
adjustable parameter, except at kernel compile time

How to run Linux program as another user

To run a script as some other user:
        /bin/su - <username> -c <script or command>

How to compare 2 directories using Unix diff command

Sometimes I am working on 2 different branches and want to compare them just to see how far off they are from each other. I found the unix diff command pretty handy for this

Following is the command
diff --recursive --exclude=.svn branch1 branch2

I have customized the output for my taste as
diff --minimal --recursive --exclude=.svn --exclude=log  branch1 branch2 \
| sed 's/diff --recursive /\n::::::::::::::::::::::::::::::\n: STARTING NEW FILE\n::::::::::::::::::::::::::::::\n&/g' \
| sed 's/Only in /\n::::::::::::::::::::::::::::::\n: STARTING NEW FILE\n::::::::::::::::::::::::::::::\n&/g'

The diff command is very powerful, here is full documentation
http://linux.about.com/library/cmd/blcmdl1_diff.htm