UNIX: Terminology I

User Process

A user process is a running instance of some user program (which must have been resident in the filesystem somewhere, such as /bin/ls or /usr/dt/bin/dtmail)



Address Space

Each user process has its own unique address space so that one user process cannot directly violate the memory space of another. The valid ranges of virtual addresses within the address spaces of two distinct processes will often look similar (e.g., executable code normally starts at address 0x10000), but will be mapped by the kernel to different physical memory addresses



Segment

An address space is made up of a number of segments. Typical segment types for a user process are text (executable code), heap (global program data), shared library (mapped in at process creation time) and stack (used to store the processes main stack). You can see the segments comprising the address space for any process using /usr/proc/bin/pmap



32-bit Program

In a 32-bit program virtual memory addresses are formulated using 32-bits, so the addressable range is 0 to 2^32 - 1 - a range of 4GB. This means that a single 32-bit process can address up to 4GB of virtual memory. There was a time when 4GB was a lot of address space, but increasingly programs are finding the need to address more than this



64-bit Program

Similarly, a 64-bit program formulates virtual memory addresses in 64-bits and has an address range of 0 to 2^64 - 1. This is an absolutely vast range of addresses - we can map very large individual segments into a 64-bit address space and we can even leave very large "holes" between the segments



32-bit Kernel

A 32-bit kernel is a kernel that uses 32-bit addresses. Being a 32-bit program, a 32-bit kernel can use at most 4GB to store all its own executable code and data structures. Since the kernel is responsible for all aspects of the system, it must maintain enormous numbers of data structures (e.g., a structure to keep track of every process created in the system, a structure to manage every physical page of memory). As systems become bigger and more complex, the 4GB that a 32-bit kernel has available in which to store its data structures has become ever more crowded



64-bit Kernel

A 64-bit kernel, on the other hand, uses 64-bit addresses and can therefore address a vast amount of memory for storing its own data structures



User Thread

Originally a user process had just a single thread of control. Execution started at the main() function and traced subsequent code and function calls in a single path. To perform tasks in parallel, a process would fork() a child to perform some work



More recently user programs have become multithreaded multiple threads of control. Execution still starts at main() but the process can create additional threads of control through calls to an API. The resulting "user level threads" can perform tasks in parallel, and even run simultaneously in a multi-CPU system



A user process is, therefore, comprised of a number (perhaps only 1 in the single-threaded case) of user threads. These user threads all share the address space of the process within which they reside. Multithreaded applications

usually employ from two to several tens of threads, but some applications are also written to use hundreds or thousands of threads



Kernel Thread

Modern UNIX kernels are also multithreaded, meaning that we have multiple threads of control within the kernel. A typical kernel will have created hundreds or even thousands of kernel threads.



Some kernel threads exist only to support the system calls made by user-level threads. When a user process (i.e., some thread in that process) requires a service of the system (necessarily provided by the kernel) it performs a system call into the kernel, and one of the kernel threads created to support that process performs the requested service



The kernel threads that support a particular process also perform some transparent work on behalf of the process For example, if a process accesses an address in a page that has been paged out to swap this will generate a page fault. Handling this page fault (page in from swap) requires the kernels intervention, and the kernel thread supporting the user thread that "pagefaulted" will perform the necessary work - the user thread will later resume without even knowing that a pagefault took place



Some kernel threads are "pure" kernel threads. These don't perform services directly for user processes, but instead perform background and housekeeping tasks. Examples are pageout, fsflush and the kernel RPC threads



Stack Frame

In C every function that is called has a corresponding stack frame (except for so-called leaf functions where the compiler can sometimes optimize the stack frame away). This provides storage for the CPU registers in use in that function (mostly we'll work within the CPU registers themselves, but at times such as when switching to a new process to run or calling another function from within the current function we may need somewhere to store the current register values). The stack frame also provides storage for the local variables of the function



Not all stack frames are the same size - they vary depending on the number and size of local variables in the function. The minimum stack frame size (one which just provides storage for registers but no local variable storage) is usually less than 100 bytes



Stack

Abstractly, a "stack" is a linear list from which insertions and deletions are made from only one end



The stack for a given thread is a linear list of stack frames. As a function call is made a new stack frame is allocated and inserted at the bottom of the stack. When the function call returns (perhaps after having made further function calls) its stack frame is removed from the bottom of the stack



Stack frames that are logically adjacent in the stack (i.e., the stack frames for two functions one of which has called the other) are usually physically adjacent in virtual memory (the processor instructions that manipulate stack pointers simply increment or decrement the current stack pointer). This means that if we have an area of memory, say one 8K page, allocated to hold a particular thread's stack and have the two adjacent pages (one above and one below this stack page) in use for other purposes that we cannot easily grow the stack outside of the 8K page it started in



User Stack

For a single-threaded process the stack resides within the stack segment of the process address space. The stack segment usually starts out reasonably small (8K) and the initial stack frame (for main) is allocated at the top of this



If during process execution the stack grows to the extent that we will "drop off" the bottom of the stack segment (remember that consecutive stack frames are usually allocated in adjacent memory ranges) the kernel can catch this access and quickly increase the stack segment size (provided no ulimit has been exceeded)



In order to allow for the possibility of stack segment growth, the virtual address range of the initial stack segment is chosen so as to have a virtual address space "hole" below it into which we can grow the stack segment. If instead the initial stack segment were placed immediately adjacent to another address space segment we would not be able to grow the stack segment



For multithreaded user processes, each thread is allocated its own stack at the time it is created (the process starts with one thread and can create others from there). Unlike the single-threaded case, we are unable to dynamically grow the stacks of these threads. The reason is quite simple - leaving the necessary virtual address

space "hole" below each allocated stack into which we could grow can soon exhaust the 32-bit address space (4GB) that a 32-bit process can access



Kernel Stack

All the kernel threads within the kernel share the same address space (just like all user threads within a single multithreaded process share an address space)



Each kernel thread is allocated its own stack at the time it is created. It immediately becomes obvious that it will be difficult to space these stacks in a 32-bit address space in such a way that there is room for growth beneath

every stack while still leaving much room for the kernel to store other material



"Pure" kernel threads always run on their allocated (kernel) stack. User threads run on their allocated stack until such time as they make a system call or until the kernel handles something like a pagefault on their behalf. At this point we switch to running the kernel thread using its allocated stack. When the system call or whatever is complete we return to running the user thread on its stack



source:

sunsolve.sun.com documents

Read More

C++: Name Mangling

When C++ compilers compile a C++ program, it encodes all function names and

certain other identifiers to include type and scoping information. This encoding

process is called name mangling. Linker uses these mangled names to ensure

type-safe linkage. These mangled names appear in the object files and final

executable file.



What's a symbol?

In every C++ program/library/object file, all non-static functions are represented

in the binary file as symbols. These symbols are special text strings that uniquely

identify a function in the program, library or object file



The Need for Name Mangling:

C language programs does not use name mangling, because in C no two non-static

functions can have the same name. i.e., the symbol name is the same as the

function name: the symbol of myfunc will be myfunc



Because C++ allows overloading (different functions with the same name but

different number of arguments) and has many features C does not, like classes,

member functions, exception specifications — it is not possible to simply use the

function name as the symbol name. To solve that, C++ uses name mangling, which

encodes the function name and all the necessary information (like the number

and size of the arguments) into some special string which only the compiler

knows about



eg.,

bpte4500s001:/sunbuild1/giri/testcases/% nm hide.o



hide.o:



[Index]   Value      Size    Type  Bind  Other Shndx   Name

[3]     |        16|      56|FUNC |GLOB |3    |2      |__1cKCRectangleKset_values6Mii_v_

[4]     |         0|       0|NOTY |GLOB |0    |ABS    |__fsr_init_value

[1]     |         0|       0|FILE |LOCL |0    |ABS    |hide.cpp

[2]     |        88|      32|FUNC |GLOB |2    |2      |main



"__1cKCRectangleKset_values6Mii_v_" is the mangled name



But this kind of scheme is undesirable for the developers because the names

are difficult to read & debug



Two utilities are available with Sun Studio C/C++ compiler collection to convert

the mangled names to their original source code names:

1)  c++filt &

2) dem



C++filt is a filter that demangles (decodes) mangled names.

bpte4500s001% echo __1cKCRectangleKset_values6Mii_v_ | c++filt

void CRectangle::set_values(int,int)



"dem" is another utility to demangle C++ names

bpte4500s001% dem __1cKCRectangleKset_values6Mii_v_

__1cKCRectangleKset_values6Mii_v_ == void CRectangle::set_values(int,int)



Note:

C++ standard does not define how names have to be mangled; thus every compiler

mangles names in its own way. Some compilers even change their name mangling

algorithm between different versions. This could be a problem if the developers

hack & rely on how compiler mangles the C++ symbols, as the same algorithm may

not work with the next version of C++ compiler

Read More

UNIX/C: cscope - an interactive program examiner

cscope is an interactive program that locates specified elements of code in C,

C++, lex, or yacc source files. cscope  is a very useful utility to browse through

the source code tree of a large project. It is a text-mode screen-oriented utility.



When cscope is called for a set of C, C++, lex, or yacc source files, it builds a

symbol cross-reference table for the functions, function calls, macros, variables,

and preprocessor symbols in those files. You can then query that table about the

locations of symbols you specify.



At First, cscope presents a menu and asks you to choose the type of search you

would like to  perform. You may, for instance, want cscope to find all the

functions that call a specified function. When it completes the search, cscope

prints a list.  Each list entry contains the name of the file, the number of the line,

and the text of the line in which cscope has found the specified code. In our case,

the list also includes the names of the functions that call the specified function.

We now have the option of requesting another search or examining one of the

listed lines with the editor. If we choose the latter, cscope invokes the editor (vi)

for the file in which the line appears, with the cursor on that line. We can now

view the code in context and, if we wish, edit the file as any other file. By simply

using vi editor's edit keys we can either navigate through the code or return back

to the menu to request a new search.



Lets try building symbol cross-reference table & search for some symbols using

cscope



bpte4500s001:/sunbuild1/giri/testcases/cscope/%ls

hidden.cpp    hidden.h      hide.cpp      htest.cpp     test.h

testcase.cpp  x.cpp         x.h



bpte4500s001:/sunbuild1/giri/testcases/cscope/%which cscope

/home/sundev/S1S9/SUNWspro/bin/cscope

^^^^

cscope is part of Sun Studio Compiler Collection 



By default, cscope builds a symbol cross-reference table for all the C, C++, lex,

and yacc source files in the current directory, and for any included header files in

the current directory or the standard place. So, if all the source files for the

program to be browsed are in the current directory, and if its header files are

there or in the standard place, invoke cscope without arguments



bpte4500s001:/sunbuild1/giri/testcases/cscope/%cscope

---------------------------------------------------------------------

cscope                                       Press the ? key for help









Find this C symbol:

Find this global definition:

Find functions called by this function:

Find functions calling this function:

Find this text string:

Change this text string:

Find this egrep pattern:

Find this file:

Find files #including this file:

---------------------------------------------------------------------



cscope builds the symbol cross-reference table the first time it is used on the

source files for the program to be browsed. By default, the table is stored in the

file "cscope.out" in the current directory. On a subsequent invocation, cscope

rebuilds the cross-reference only if a source file has been modified or the list of

source files is different



bpte4500s001:/sunbuild1/giri/testcases/cscope/%ls

cscope.out    hidden.cpp    hidden.h      hide.cpp

htest.cpp     test.h        testcase.cpp  x.cpp         x.h



bpte4500s001:/sunbuild1/giri/testcases/cscope/%file cscope.out

cscope.out:     data



bpte4500s001:/sunbuild1/giri/testcases/cscope/%cat hidden.h

__declspec(dllexport) char *lastname(char *);

//char *lastname(char *);

__declspec(dllexport)

int agefunc();

__declspec(dllexport)

char firstchar(char *);



Now lets try to locate the function "lastname" using cscope



---------------------------------------------------------------------

C symbol: lastname



File     Function   Line

1 hidden.h __declspec 1 __declspec(dllexport) char *lastname(char *);







Find this C symbol:

Find this global definition:

Find functions called by this function:

Find functions calling this function:

Find this text string:

Change this text string:

Find this egrep pattern:

Find this file:

Find files #including this file:

---------------------------------------------------------------------



After cscope shows you the results of a successful search, you have several

options. You may want to change one of the lines or examine the code

surrounding it in the editor. Or, if cscope has found so many lines that a list of

them does not fit on the screen at once, you may want to look at the next part of

the list.



If you want to open up "hidden.h", just press 1; it opens up "hidden.h" in vi

editor. If more than one file has the designated symbol, we just need to press

the number that is besides the file name which we want to edit. If the search

yields more results and if all the results doesn't fit in a single screen, we need

to press "space bar" to go to the next screen. To go back to the previous screen,

we need to press "--"



NOTE:

If the source code of a project is spread over different directories we need to get

the list of files (with the path) and save the list in a file called "cscope.files".

Then run cscope to build the symbol cross-reference table & to show the menu.

Once the symbol cross-reference table is ready & if there are no changes in the

source code, invoke cscope with "-d" flag from subsequent executions. "-d"

instructs cscope not to update the cross-reference



bpte4500s001:/sunbuild1/giri/testcases/cscope/%ls -ld *

drwxrwxr-x   2 build    engr         512 Nov 15 16:44 hidden

-rw-r--r--   1 build    engr          79 Nov 15 16:18 htest.cpp

drwxrwxr-x   2 build    engr         512 Nov 15 16:44 test

drwxrwxr-x   2 build    engr         512 Nov 15 16:44 x



bpte4500s001:/sunbuild1/giri/testcases/cscope/%ls -lR *

-rw-r--r--   1 build    engr          79 Nov 15 16:18 htest.cpp



hidden:

total 6

-rw-r--r--   1 build    engr         320 Nov 15 16:18 hidden.cpp

-rw-r--r--   1 build    engr         155 Nov 15 16:18 hidden.h

-rw-r--r--   1 build    engr         358 Nov 15 16:18 hide.cpp



test:

total 4

-rw-rw-r--   1 build    engr          42 Nov 15 16:18 test.h

-rw-rw-r--   1 build    engr         354 Nov 15 16:18 testcase.cpp



x:

total 4

-rw-r--r--   1 build    engr          60 Nov 15 16:18 x.cpp

-rw-r--r--   1 build    engr          47 Nov 15 16:18 x.h



bpte4500s001:/sunbuild1/giri/testcases/cscope/%cscope

cscope: no source files found



bpte4500s001:/sunbuild1/giri/testcases/cscope/%find . -name "*" > cscope.files



bpte4500s001:/sunbuild1/giri/testcases/cscope/%ls

cscope.files  hidden        htest.cpp     test          x



bpte4500s001:/sunbuild1/giri/testcases/cscope/%cscope

(.. skipped cscope menu ..)



bpte4500s001:/sunbuild1/giri/testcases/cscope/%ls

cscope.files  hidden        test

cscope.out    htest.cpp     x



Note:

If cscope throws an error message like "Input string too long, limit 50251",

run truss on cscope and observe the last few lines to find the culprit and remove

it from cscope.files to let cscope continue building the symbol database



Courtesy:

Sun Microsystems, Sun Studio 9: C User's Guide

Read More

Sun C/C++: Installed Components & Version#s

bpte4500s001:/sunbuild2/web/dump/%which version

/sunbuild3/s1s9/SUNWspro/prod/bin/version



bpte4500s001:/sunbuild2/web/dump/%version

Machine hardware:   sun4u

OS version:         5.8

Processor type:     sparc

Hardware:           SUNW,Ultra-Enterprise



The following components are installed on your system:





Sun Studio 9

      Sun Studio 9 C Compiler

      Sun Studio 9 C++ Compiler

      Sun Studio 9 Tools.h++ 7.1

      Sun Studio 9 C++ Standard 64-bit Class Library

      Sun Studio 9 Garbage Collector

      Sun Studio 9 Fortran 95

      Sun Studio 9 Debugging Tools (including dbx)

      Sun Studio 9 Debugger GUI

      Sun Studio 9 Performance Tools (including collect, ...)

      Sun Studio 9 X-Designer

      Sun Studio 9 VIM editor

      Sun Studio 9 XEmacs editor

      Sun Studio 9 Native Connector Tool

      Sun Studio 9 LockLint

      Sun Studio 9 Building Software (including dmake)

      Sun Studio 9 Documentation Set



version of "/sunbuild3/s1s9/SUNWspro/prod/bin/../../bin/cc": Sun C 5.6 2004/06/02

version of "/sunbuild3/s1s9/SUNWspro/prod/bin/../../bin/CC": Sun C++ 5.6 2004/06/02

version of "/sunbuild3/s1s9/SUNWspro/prod/bin/../../bin/f90": Sun Fortran 95 8.0 2004/06/02

version of "/sunbuild3/s1s9/SUNWspro/prod/bin/../../bin/dbx": Sun Dbx Debugger 7.3 2004/06/02

version of "/sunbuild3/s1s9/SUNWspro/prod/bin/../../bin/analyzer": Sun Performance Analyzer 7.3 2004/06/02

version of "/sunbuild3/s1s9/SUNWspro/prod/bin/../../bin/dmake": Sun Distributed Make 7.5 2004/06/02

Read More

JDS: Linux & Sony DSC-V1/W1 Digital Camera

summary:



switch the camera in USB PTP (Picture Transfer Protocol) mode and then run 

"gtkam" application



1) Setup the camera to enable PTP by default



      i) unplug USB cables (if any) from camera

     ii) turn the dial to "setup"

    iii) using arrow keys navigate to USB connect & set the mode to 

        "PTP" ("normal" is the default)

     iv) exit the setup

Quick test:

     i) plug-in the USB cable from Camera to computer

    ii) turn on the camera <= it should read "USB Mode - PTP". If it shows 

        "USB Mode - Normal", we have to go back to step (1) and enable 

        PTP



2) Run "gtkam" on Linux machine



    i) Install the latest versions of libgphoto2, gPhoto2, and gtkam on 

        your machine, if not installed already. JDS install gtkam & 

        the dependencies (libraries) by default, unless you choose 

        not to install during installation of OS



    URL for gtkam, gPhoto2 & libgphoto2: http://www.gphoto.org/download/



    ii) Run "gtkam" by typing gtkam& in a terminal window and go to the 

        Camera -> Add Camera. Select "Sony DSC-V1" or "Sony DSC-W1" 

        camera from the list of cameras and press OK



    iii) It should quickly initialize the camera and then add it to the 

        camera listing on the left in the main window of gtkam. Browse 

        through the tree of the camera and select any folders that you 

        have setup on the camera



    iv) You can right click on any picture or movie and select Save or select 

        multiple pictures and movies and goto the File menu and select 

        Save selected Photos or use Ctrl+S as a shortcut to save the 

        photos and movies to your hard drive



    v)  Play with options "Save photos, Save raw data, Save audio data, 

        Save thumbnails, & Save EXIF data" and choose the format in 

        which you want to save the data from Camera



3) Other recommended softwares



Probably want to add many more things to your Linux system to deal with all of 

the data you will downloading from your camera. Following are some of the 

recommended softwares:



    * Eye of Gnome image viewer for looking at all of your photos

        URL: http://www.gnome.org/gnome-office/eog.shtml



    * GIMP image editing software to edit and manipulate all of your photos 

        (e.g. remove blemishes, take out red eye, etc.,)

        URL: http://www.gimp.org/downloads/



        ** Red Eye Removal tutorial for GIMP

            URL: http://www.gimp.org/





        ** EXIF Browser Plug-in for GIMP so that you can view all of the 

            important information stored in your photos (e.g. date, 

            time, resolution, camera settings, etc.). This will need 

            the libexif library to be install as well



            EXIF Browser Plug-in: http://registry.gimp.org/plugin?id=4153

            libexif library: http://sourceforge.net/projects/libexif



    * jhead is a console utility that will allow you to view all of the detailed 

        EXIF information saved to your photos by the camera

        URL: http://www.sentex.net/~mwandel/jhead/



    * MPlayer to play any of the movies created with your camera. This will need 

        a few things to be installed first: among other things are the LAME 

        libraries



        MPlayer: http://www.mplayerhq.hu/homepage/dload.html

        LAME libraries: http://lame.sourceforge.net/download/download.html



Acknowledgements:

    John Wyles

Read More