Siddesh BG's Build Release Config mgmt Blog

  • Subscribe to our RSS feed.
  • Twitter
  • StumbleUpon
  • Reddit
  • Facebook
  • Digg

Tuesday, 17 May 2011

Developing packages with RPM

Posted on 09:45 by Unknown
Reference: Max RPM book, Edward C. Bailey

Building a package is similar to compiling code—there are inputs, an engine, and outputs.
Inputs
1) Sources - It should be a tar file. RPM can handle other archive formats, but a bit more up-front effort is required.
2) Patches - RPM gives you the ability to automatically apply patches.
3) Spec File - It contains information required by RPM to build the package, as well as instructions telling RPM how to build it. It also dictates exactly what files are a part of the package, and where they should be installed. There are eight sections in spec file
  3.1 - The Preamble  - It contains information that will be displayed when users request information about the package. This would include a description of the package’s function, the version number of the software, and so on.
  3.2 - The Prep Section - necessary preparations are made prior to the actual building of the software. The contents of this section are an ordinary shell script. However, RPM does provide two macros. One macro can unpack a compressed tar file and cd into the source directory. The other macro easily applies patches to the unpacked sources.
  3.3 - The Build Section - The build section consists of a shell script. It is used to perform whatever commands are required to compile the sources. This section could consist of a single make command, or be more complex if the build process requires it.
  3.4 - The Install Section - It also consists of a shell script. It is used to perform the commands required to install the software. This section might only consist of a make install command. Otherwise, the usual assortment of cp, mv, or install commands to get the job done.
  3.5 - Install and Uninstall Scripts - It consists of scripts that will be run, on the user’s system, when the package is actually installed or removed.
  3.6 - The Verify Script - It is executed when RPM verifies the package’s proper installation. While RPM does most of the work verifying packages, this script can be used to verify aspects of the package that are beyond RPM’s capabilities.
  3.7 - The Clean Section - contains a script that can clean things up after the build. This script is rarely used, since RPM normally does a good job of clean-up in most build environments.
  3.8 - The File List - The last section consists of a list of files that will comprise the package. Additionally,
a number of macros can be used to control file attributes when installed, as well as to denote which files are documentation, and which contain configuration information. The file list is very important — if it is missing, no package will be built.

Engine: RPM
It performs a number of steps during the build process:
  *) Executes the commands and macros in the prep section of the spec file.
  *) Checks the contents of the file list.
  *) Executes the commands and macros in the build section of the spec file.
  *) Executes the commands and macros in the install section of the spec file. Any macros in the file list are executed at this time, too.
  *) Creates the binary package file.
  *) Creates the source package file.
     Outputs
     The end product of this entire process is a source package file and a binary package file.
      1) Source package: It is a specially formatted archive that contains the following files - the original compressed tar file(s), spec file, patches. It is a great way to archive all the information needed to rebuild a particular version of the package.
      2) Binary RPM: It contains the files that comprise the application, along with any additional information needed to install and erase it.

    Building packages
    Step 1: Creating the Build Directory Structure
      Default directory layout consists of a single top-level directory (/usr/src/redhat) with five subdirectories.
      /usr/src/redhat/
            SOURCES  -  Contains the original sources, patches, and icon files.
            SPECS - Contains the spec files used to control the build process.
            BUILD - The directory in which the sources are unpacked, and the software is built.
            RPMS - Contains the binary package files created by the build process.
            SRPMS - Contains the source package files created by the build process.
    Read More
    Posted in rpm | No comments

    Friday, 8 April 2011

    Fortify scan automation steps for analyzing c/c++ code (Makefiles)

    Posted on 04:29 by Unknown
    I wrote in my previous blog about installing and configuring Fortify client. This blog presents standard steps to automate fortify scan for c/c++ code which are compiled using Makefiles.

    Step 1: Compile your source code by instrumenting Fortify
          Normally we compile source code using  compilers like cc, gcc, cl.exe or devenv. To instrument fortify append sourceanalyzer (fortify tool) to your compilation command at the beginning.
         For ex: sourceanalyzer -b testing-fortify cc test.c

         This command will compile test.c and generates NST file, which is understood by Fortify tool.
          Note: .nst files can be located at $HOME/.fortify 

         In most cases we don't compile individual files like it shown above. We will be using Makefiles to manage compilation. In that case we need to inform Make to call sourceanalyzer at the time compilation.
        If our Makefiles (usually top level makefiles) have defined the CC variable, then we can modify it as given below
        ifdef FORTIFY
            CC="sourceanalyzer -b MyProject $CC"
        endif

        With this definition, your Make command will be able to compile all your source files using sourceanalyzer.

    Step 2: Scan NST files to generate fpr file
        Fortify generates a fpr file using the NST files generated in step 1. Once all your files are compiled in step 1, you need to run this step only once to generate one combined FPR file. This FPR file will be understood by other fortify tools used for reporting.

       sourceanalyzer -b MyProject -scan -f MyProject.fpr

      This will generate a FPR file named myproject.fpr which will be used in next steps.

    Step 3: Upload the FPR file to Fortify 360 server
       Fortify 360 server is web based tool, which displays fortify scan result. The input to this tool is the FPR file which we generated in Step 2.
       We can upload the FPR file to Fortify 360 server using the command given below

        fortifyclient -url http://my-fortify-360-server:8282/f360 -authtoken afknafowqnewksdgjsgddkg  uploadFPR -file MyProject.fpr -project MyProject  -version 1.0

      where for
          -authtoken : You need to generate authentication token for login to Fortify 360 server. Refer my previous blog to know how to generate it
          -project: You need to create your project name in your Fortify 360 server, prior to this step.
          -version: You need to create your project version in your Fortify 360 server, prior to this step.

    Step 4: Generating PDF report using the FPR file
       You can generate a PDF or XML report out of FPR file, which can be sent through mail for developers.
       Here is the command to do it
        ReportGenerator -format pdf -f MyProject.pdf -source MyProject.fpr

        Refer my previous blog for detailed information about this step.

     This completes the automation steps for Fortify scan on c/c++ code.


    Note: You can use an application called auditworkbench to analyze fortify scan report. Even input for auditworkbench is FPR file.

    Some more useful commands
     - Use sourceanalyzer -b MyProject -show-files to know what all files are associated with the tag MyProject
     - Use sourceanalyzer -b MyProject -show-build-warnings to show errors and warnings
     - Use sourceanalyzer -b MyProject -show-loc to show Lines of code. 
    Read More
    Posted in Fortify | No comments

    Tuesday, 22 March 2011

    Perforce streams - a new feature in Perforce (not yet released)

    Posted on 04:29 by Unknown
    Perforce is coming up with a new feature called "streams". Not mentioned when it will be released. But they are promoting that, this feature will enhances branch management and reduces merge issues.

    What is a stream?
                    A stream describes about branch like depot location, owner, it's parent branch and more. It is a branching and merging application based on the perforce system.

    Notable features:
    ·         Streams provide a centralized control to manage project related workspaces. Just setup one stream view and link your workspace to that stream.
                    Whenever stream view changes, it automatically update workspace views.
    ·         Streams can be used to compute sensible defaults for merges
    ·         P4V is being enhanced to show streams in a slick visual flowchart, and to offer a simple interface for sophisticated stream tasks.

    The current available details about this feature can be obtained from these links
    http://blog.perforce.com/blog/?p=2948
    &
    http://blog.perforce.com/blog/?p=2951
    and
    Streams a Tiny tutorial
    Read More
    Posted in package | No comments

    Tuesday, 28 December 2010

    Installing and configuring Fortify on Linux and Windows machines

    Posted on 03:27 by Unknown
    Installing Fortify on Linux (RHEL 5 32 bit)
    • Download Fortify archive Fortify-360-2.6.5-Analyzers_and_Apps-Linux-x86.tar.gz and extract it to a directory like /usr/local/fortify
    • Get License file fortify.license and place it under root directory (/usr/local/fortify)
    • Run utility scapostinstall under bin directory (/usr/local/fortify/bin) to perform some necessary post install activities.
            ./scapostinstall
               [1] Migration...
               [2] Settings...
               [s] Display all settings
               [q] Exit
               Please select the desired action (1,2,s,q):

              Give valid entries for  Rulepack Update and 360 Server Settings

              Select 2 and proceed for changing settings. In Rulepack Update, give http address of your fortify 360 server for  Update Server URL: field. In 360 Server Settings, give again http address of your fortify 360 server for Server URL: field and set Get Rulepack Updates from 360 Server: true

    • Now update rules pack using tool rulepackupdate available at bin directory (/usr/local/fortify/bin) 
    • Generate Upload access token using utility fortifyclient under bin directory. The upload access token enable account and password information to be concealed during uploading of FPRs to Fortify 360 Server          
              fortifyclient -url [360_server_URL] token -gettoken AnalysisUploadToken -user [AccountName]
     
              fortifyclient prompts for a password, type the password for [AccountName]. fortifyclient displays a token of the general form cb79c492-0a78-44e3-b26c-65c14df52e86. Copy the token returned by fortifyclient into a text file.

    Updating rules in future

    • In case in future if you want to update rule packs, here is the process
    • You may get rules pack in a zip file of the form Se289787b-abd8-4ad6-a77d-f11d89e8ac60.zip 
              Then run the command
             /usr/local/fortify//bin/rulepackupdate -import Se289787b-abd8-4ad6-a77d-f11d89e8ac60.zip
    • or if your F360 server is up to date with rules, then run the command
               /usr/local/fortify//bin/rulepackupdate -url  [360_server_URL]
    Read More
    Posted in Fortify | No comments

    Wednesday, 15 December 2010

    Fortify report templates using ReportGenerator

    Posted on 01:56 by Unknown
    Fortify Static Code Analysis Tool allows us to create scan reports using command line utility ReportGenerator.
    By default ReportGenerator creates report using the template OWASP2007.xml
    Here is an example of generating PDF scan report using command line utility
    ReportGenerator -format pdf -f outputFile.pdf -source dev-rkm-KMS-aggregate.fpr
    We can create report either in pdf or rtf or xml.

    Some times we get an error like
    Xlib: connection to "localhost:10.0" refused by server
    Xlib: PuTTY X11 proxy: wrong authentication protocol attempted


    It means ReportGenerator will open Xwindows and your server doesn't have any Xserver running. You can try to run some simple X applications like xcalc or xterm on your machine and make sure Xserver is running fine.

    But there is also an option -template, using which we can generate reports of various formats.This option is not well documented. When you say "ReportGenerator -help", it just says
    -template       The Fortify Report template used to define the report.

    But what are the various available template names, which they are not giving information.

    Anyways still we can find out available templates in the directory fortify-install-dir/Core/config/reports
    Available templates are
    1) DefaultReportDefinition.xml  
    2) DeveloperWorkbook.xml 
    3) OWASP2004.xml  
    4) OWASP2007.xml  
    5) ScanReport.xml

    Here is an example of using template option
    ReportGenerator -format pdf -f outputFile.pdf -source dev-rkm-KMS-aggregate.fpr -template "ScanReport.xml"
    Read More
    Posted in Fortify | No comments

    Sunday, 21 November 2010

    How to invoke xterm from putty?

    Posted on 23:45 by Unknown
    When we connect to a Linux/Solaris or any Unix servers using putty from Windows machine and try to access any X applications like xterm, xeyes, etc it may throw errors like

    Xlib: connection to "localhost:10.0" refused by server
    Xlib: PuTTY X11 proxy: wrong authentication protocol attempted
    Error: Can't open display: localhost:10.0
    or
    X connection to localhost:10.0 broken (explicit kill or server shutdown)

    Then you need to install Xming on your Windows machine. Xming is the X Window Server for Microsoft XP/2008/Windows7.

    Start Xming in your windows machine and now login to your Unix machine through putty
    For Ex:
    #ssh root@spodumene.ap.rsa.net
    [root@spodumene build]# xeyes

    Here xeyes is a simple X application, which displays a pair of eyes. If it displays this, then you can start any  X applications.

    But still you may get issues if you login as root and then do "su - user" and try to access X application.

    [root@spodumene build]# su - build
    [build@spodumene ~]$ xeyes
    Xlib: connection to "localhost:10.0" refused by server
    Xlib: PuTTY X11 proxy: wrong authentication protocol attempted
    Error: Can't open display: localhost:10.0

    You need to temporarily transfer the authorization to the other account. First, get the key from your account
    To fix this, do following. Login as root and call "xauth list" command.
    [root@spodumene build]# xauth list
    spodumene.ap.rsa.net/unix:12  MIT-MAGIC-COOKIE-1  e0190c6d94addb5201f3d8cbeef32b72
    spodumene.ap.rsa.net/unix:13  MIT-MAGIC-COOKIE-1  b1e7ee620ddef216e32cff36945a31a0
    spodumene.ap.rsa.net/unix:11  MIT-MAGIC-COOKIE-1  5db41fd092612581a408a762e252494b
    spodumene.ap.rsa.net/unix:10  MIT-MAGIC-COOKIE-1  bb6f1099c09e630fd5caed5dc7b8d143


    Now do "su" to your account
    [root@spodumene build]# su - build

     And add auth tokens listed in the previous command using "xauth add" command.
    [build@spodumene ~]$ xauth add spodumene.ap.rsa.net/unix:10  MIT-MAGIC-COOKIE-1  bb6f1099c09e630fd5caed5dc7b8d143


    You can repeat this for remaining tokens. After that run "xauth list", which will display all the added tokens.
    [build@spodumene ~]$ xauth list
    spodumene.ap.rsa.net/unix:10  MIT-MAGIC-COOKIE-1  bb6f1099c09e630fd5caed5dc7b8d143
    spodumene.ap.rsa.net/unix:12  MIT-MAGIC-COOKIE-1  e0190c6d94addb5201f3d8cbeef32b72
    spodumene.ap.rsa.net/unix:13  MIT-MAGIC-COOKIE-1  b1e7ee620ddef216e32cff36945a31a0
    spodumene.ap.rsa.net/unix:11  MIT-MAGIC-COOKIE-1  5db41fd092612581a408a762e252494b

    Now run your X application
    [build@spodumene ~]$ xterm

    It will open an xterm.
    Read More
    Posted in xterm | No comments

    Wednesday, 22 September 2010

    Silent Installation and Uninstallation using setup.exe - Installshield

    Posted on 05:23 by Unknown
    Silent installation and uninstall is a necessary requirement of automation process. In our organization we have a build dependency where one build is (c-sharp) dependent on installation of another build (C client build). Here we need to install a build package generated using installed silently and uninstall it once build completes.

    Since InstallShield generated "setup.exe", there is an easy way to do silent installation. But I found the available solutions to silent uninstall is not working well for our product. Anyways I will give my findings.

    Silent Installation
     - First create a InstallShield silent response file(setup.iss). This file will record the actions which we do while installing
          setup.exe -r
       It will create setup.iss file in Windows folder. i.e C:\WINNT in my case.
       Also you can use "-f1" option to create .iss file of your choice.
       For Ex: setup.exe -r -f1"C:\win32vc8\setup_inst.iss"
    - Now Install with silent option (-s or /s)
       setup.exe -s
            or
       setup.exe -s -f1"C:\win32vc8\setup_inst.iss"
       If everything go well, then application will be installed and can be accessed from Program files.
    - Trouble shooting errors
      Look setup.log file and make sure "ResultCode=0". By default setup.log file will be created at location where setup.exe is located. We can change this default location be providing alternative path with -f2 option.
      What various non-zero ResultCode value means?
       0 Success
       -1 General error
       -2 Invalid mode
       -3 Required data not found in the Setup.iss file
       -4 Not enough memory available
       -5 File does not exist
       -6 Cannot write to the response file
       -7 Unable to write to the log file
       -8 Invalid path to the InstallShield Silent response file
       -9 Not a valid list type (string or number)
       -10 Data type is invalid
       -11 Unknown error during setup
       -12 Dialogs are out of order
       -51 Cannot create the specified folder
       -52 Cannot access the specified file or folder
       -53 Invalid option selected


    Interactive Uninstall from command line.
    setup.exe /uninst

    Silent Uninstall.  This is what worked for me.
    Two ways
    1) Using a Response File
       To run an uninstallation using a response file:
    1. Prepare a response file for the uninstallation (.iss) by running Setup.exe with the /r argument: Setup.exe /r
    2. Locate the Setup.iss file generated in the Windows folder and copy it to the desired location.
    3. Type the following at the command line (items in Italics represent data that is specific to your product's uninstallation): IDriver.exe /M{Your Product GUID} /s /f1"<FULLY qualified path>\YourResponseFile.iss"
    Note: The /f1 parameter is necessary only if the Setup.iss file is located in a different directory than the Setup.exe file.

    2) Simple Uninstallation
    If you do not want to follow the script logic and want to uninstall the product, you can use the following command line:
    IDriver.exe /M{Your Product GUID} /uninst
    The /uninst parameter causes a forced uninstallation without opening the script. It rolls back the system changes made during the installation, including those from the MSI package and any InstallShield scripting.
    Note: For both of these uninstallation options, the /M argument is case sensitive.
    Note: By default, the file "IDriver.exe" is located in the following path: C:\Program Files\Common Files\InstallShield\Driver\\Intel 32.


     This approach silently removed the application entry from Windows registry ( regedit). But it didn't remove the installed files.

    If you wish to create Response file manually, here is the link for instructions http://kb.flexerasoftware.com/doc/Helpnet/installshield12helplib/CreatetheResponseFile.htm
    Read More
    Posted in InstallShield | No comments
    Newer Posts Older Posts Home
    Subscribe to: Posts (Atom)

    Popular Posts

    • 0509-036 Cannot load program p4 because of the following errors
      Here is the full description of error ............ bash-3.00# p4 info exec(): 0509-036 Cannot load program p4 because of the following error...
    • fortifyclient uploadFPR An internal error has occurred
      When you try to upload a .fpr file to Fortify 360 server and you get the below mentioned error. Then, this blog provides one of the route ca...
    • Unable to resolve target system name - a DNS problem
      I was not able to ping to any machines from my Windows 2003 server. I did following steps to debug & resolve the issue, which was relate...
    • jenkins error: java.io.IOException: Authentication method password not supported by the server at this stage
      When I tried to add a node to jenkins/hudson using ssh as launch method, the authentication keeps on failing with the below error. [12/15/11...
    • Posting a JIRA bug using Perl Mechanize
      Perl provides modules which can be used as command line browser to automate tasks dependent on web pages. Among them LWP and mechanize are i...
    • Installing and configuring Fortify on Linux and Windows machines
      Installing Fortify on Linux (RHEL 5 32 bit) Download Fortify archive Fortify-360-2.6.5-Analyzers_and_Apps-Linux-x86.tar.gz and extract it to...
    • Perforce and cygwin
      Are you a command-line freak ? Do you want your automated shell scripts to run on Windows ? Do you wish to work with Perforce commands on Cy...
    • Perforce - can't edit exclusive file already opened
      In perforce, whenever a binary file like doc, xls or ppt files are checked out, it is opened in exclusive lock mode. So no other person can ...
    • Solution to Project Euler Problem 10 - Find the sum of all the primes below two million
      http://projecteuler.net/problem=10 Problem The sum of the primes below 10 is 2 + 3 + 5 + 7 = 17. Find the sum of all the primes below two mi...
    • AIX: make: 1254-055 Dependency line needs colon or double
      We get this compilation issue "make: 1254-055 Dependency line needs colon or double" while compiling C/C++ code in AIX machines. I...

    Categories

    • AIX
    • AIX ssh
    • ANT
    • apache
    • appliance
    • awk
    • branching
    • build-failures
    • cgi-perl
    • code-signing
    • commands
    • continuous Integration
    • cvs
    • cygwin
    • DNS
    • Drupal
    • EPM
    • euler
    • Fortify
    • hadoop
    • hpux
    • html
    • InstallShield
    • iptables
    • iso
    • jenkins-hudson
    • Jira
    • kiwi
    • linux
    • Makefile
    • maven
    • Miscellaneous
    • mysql
    • nexus
    • NFS
    • package
    • Perforce
    • Perl
    • php
    • rbuilder
    • rpath
    • rpm
    • rsync
    • Solaris
    • ssh
    • SuseStudio
    • tinderbox
    • unix
    • Visual studio 2008
    • vmware
    • war
    • webserver
    • wget
    • windows
    • xterm

    Blog Archive

    • ▼  2013 (12)
      • ▼  December (1)
        • How to restart windows from command line ?
      • ►  July (2)
      • ►  April (2)
      • ►  March (2)
      • ►  February (3)
      • ►  January (2)
    • ►  2012 (43)
      • ►  December (2)
      • ►  November (1)
      • ►  October (4)
      • ►  September (7)
      • ►  August (5)
      • ►  July (4)
      • ►  June (2)
      • ►  May (3)
      • ►  April (4)
      • ►  March (3)
      • ►  February (1)
      • ►  January (7)
    • ►  2011 (23)
      • ►  December (4)
      • ►  November (9)
      • ►  October (4)
      • ►  September (1)
      • ►  June (2)
      • ►  May (1)
      • ►  April (1)
      • ►  March (1)
    • ►  2010 (15)
      • ►  December (2)
      • ►  November (1)
      • ►  September (3)
      • ►  April (1)
      • ►  February (6)
      • ►  January (2)
    • ►  2009 (28)
      • ►  November (5)
      • ►  October (3)
      • ►  September (2)
      • ►  August (1)
      • ►  July (1)
      • ►  June (5)
      • ►  May (3)
      • ►  April (1)
      • ►  February (2)
      • ►  January (5)
    • ►  2008 (20)
      • ►  December (6)
      • ►  November (3)
      • ►  October (1)
      • ►  September (1)
      • ►  July (8)
      • ►  June (1)
    Powered by Blogger.

    About Me

    Unknown
    View my complete profile