Showing posts with label File systems. Show all posts
Showing posts with label File systems. Show all posts

Friday 24 August 2018

Does RHEL 7 support online resize of disk partitions?

Environment

  • Red Hat Enterprise Linux 7

Issue

  • Both Xen and KVM support online resize of the virtual block devices that the virtual machines use.
  • Unfortunately, since the partitions in the boot virtual block device are already in use, those can't be modified online and the virtual machine won't pick up the changes on the boot virtual block device without a reboot.
  • We are looking for ability to dynamically increase a partitioned volume, is it possible to resize the disk partitions online in RHEL 7?
  • We are unable to dynamically increase a SAN attached volume without taking the volume offline. When will this feature to Support online resize of disk partitions be available?

Resolution

The ability to resize a Logical Volume and filesystem has long been provided by Logical Volume Manager (LVM) by simply using the entire block device (/dev/sdx for example) as a Physical Volume in the LVM Volume Group. There is no need for the old style of partition. This would allow the Logical Volume and filesystem to easily be expanded.
As to older style partitions, this feature has been added in RHEL 7 current release with a feature request (RFE has been filed to add support for online resize of disk partitions to RHEL 7 in private Red Hat Bugzilla 853105). With this feature, it's possible to resize the disk partitions online in RHEL 7.

Warning: Online partition table resizing is not a recommended process mostly because the underlying layer of an active logical volume or filesystem that relies on that information is being modified. While it is possible to perform online partition resize operation, it is not recommended simply from a data integrity standpoint and not because it's not possible.

  • In order to resize online a partition which is in use please observe the following steps:
# fdisk -l /dev/vda

Disk /dev/vda: 32.2 GB, 32212254720 bytes, 62914560 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000db7e6

   Device Boot      Start         End      Blocks   Id  System
/dev/vda1   *        2048     1026047      512000   83  Linux
/dev/vda2         1026048    28289023    13631488   8e  Linux LVM

# cat /proc/partitions 
major minor  #blocks  name

 252        0   31457280 vda
 252        1     512000 vda1
 252        2   13631488 vda2
  11        0    1048575 sr0
 253        0   10240000 dm-0
 253        1    2129920 dm-1

# pvs
  PV         VG          Fmt  Attr PSize  PFree
  /dev/vda2  rhel_vm-205 lvm2 a--  13.00g 1.20g
  • Modify the on-disk partition table as usual (e.g. by using fdisk command).
    • Delete the partition:
      Command (m for help): d
      Partition number (1,2, default 2): 2
      Partition 2 is deleted
      
    • Re-create the partition with the new size:
      Command (m for help): n
      Partition type:
         p   primary (1 primary, 0 extended, 3 free)
         e   extended
      Select (default p): p
      Partition number (2-4, default 2): 2
      First sector (1026048-62914559, default 1026048): 
      Using default value 1026048
      Last sector, +sectors or +size{K,M,G} (1026048-62914559, default 62914559): +18G
      Partition 2 of type Linux and of size 18 GiB is set
      
      Command (m for help): t
      Partition number (1,2, default 2): 2
      Hex code (type L to list all codes): 8e
      Changed type of partition 'Linux' to 'Linux LVM'
      
      Command (m for help): p
      
      Disk /dev/vda: 32.2 GB, 32212254720 bytes, 62914560 sectors
      Units = sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 512 bytes
      I/O size (minimum/optimal): 512 bytes / 512 bytes
      Disk label type: dos
      Disk identifier: 0x000db7e6
      
         Device Boot      Start         End      Blocks   Id  System
      /dev/vda1   *        2048     1026047      512000   83  Linux
      /dev/vda2         1026048    38774783    18874368   8e  Linux LVM
      
    • Commit changes to on-disk partition table:
      Command (m for help): w
      The partition table has been altered!
      
      Calling ioctl() to re-read partition table.
      
      WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
      The kernel still uses the old table. The new table will be used at
      the next reboot or after you run partprobe(8) or kpartx(8)
      Syncing disks.
      
    • While on-disk partition table has been updated, observe that on-memory kernel partition table has not:
      # partprobe 
      Error: Partition(s) 2 on /dev/vda have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use.  As a result, the old partition(s) will remain in use.  You should reboot now before making further changes.
      
      # cat /proc/partitions | grep vd
       252        0   31457280 vda
       252        1     512000 vda1
       252        2   13631488 vda2
      
  • Execute partx (provided by util-linux package) with --update option on the block device to update the in-memory kernel partition table from the on-disk partition table:
    # partx -u /dev/vda
    
  • Verify that in-memory kernel partition table has been updated with the new size:
    # cat /proc/partitions | grep vd
     252        0   31457280 vda
     252        1     512000 vda1
     252        2   18874368 vda2
    
  • Proceed with any further steps, in this example by extending the PV on the partition:
    # pvresize /dev/vda2
      Physical volume "/dev/vda2" changed
      1 physical volume(s) resized / 0 physical volume(s) not resized
    
    # pvs
      PV         VG          Fmt  Attr PSize  PFree
      /dev/vda2  rhel_vm-205 lvm2 a--  18.00g 6.20g
    

Root Cause

  • Changes to allow online resizing of in-use partitions are required in both kernel and userland.
    • RHEL7 kernels include the required changes to BLKPG ioctl from block: add partition resize function to blkpg ioctl to support BLKPG_RESIZE_PARTITION operation.
    • RHEL7 util-linux package includes partx and resizepart binaries, currently the only userland commands that support BLKPG_RESIZE_PARTITION operation of BLKPG ioctl.

Tuesday 26 July 2016

How to Create You Lab on Kali Linux - Using VMware or VBox

Before you start - Here are something that you need to consider..

  • Install VirtualBox or Vmware
  • Install LiveCD in VirtualBox or Vmware
  • Boot and try out your LiveCD

Introduction.


There are lots of tutorials available on the internet related to hacking but the big problem lies in testing your theoretical skills. Unless you don't have practical exposure to hacking, you cannot really understand the strength of it. The technique I will discuss here will be advantageous for those who have only single system and want to set up a testing network using it. In case you have multiple systems then you can easily test for your skills by making one system as target and other as attacker.
VMware Workstation 12 Pro - Leading Edge PC Virtualization VMware Workstation 12 Pro


Download VMware Workstation or Player

The best way to practice hacking is within a virtual environment. Essentially, you set up a hacking system, such as Kali Linux, and some victims to exploit. Ideally, you would want multiple operating systems (Windows XP, Vista, 7, and 8, as well as a Linux flavor) and applications so that you can try out a variety of hacks.

Virtual machines and a virtual network are the best and safest way to set up a hacking lab. There are several virtualization systems out there, including Citrix, Oracle's VirtualBox, KVM, Microsoft's Virtual PC and Hyper-V, and VMware'sWorkstation, VMware Player and ESXi. For a laboratory environment, I strongly recommend VMware's Workstation or Player. Workstation is commercial product that costs under $200, while Player is free. You can also get a free 30-day trial of Workstation.

Player is limited to just playing VMs , while Workstation can both create and play VMs. Let's download VMware's Workstation or Player here.


Then - For you Images you can download a pre-built image. As it will allow you to save some time, you don't need to build the image from scratch. Here is a link for you to download several Linux pre-built Linux Images on SourceForge.

I don't encourage you to download pre-built Windows VM from anywhere, for the single fact that Microsoft itself provides you with pre-built Windows VM for you to use on your VM Lab. Again I did the research for you and here it the link for you to download the Windows of your choice,  Obs: Remember Windows only allow you to use those VM images for 30 days. Please, READ Microsoft TERMS AND CONDITIONS before you use their software here is the url for you.



Keep posted - This tutorial continues on Part 2

Also check this Handful tutorials I have found online:

Checkout our tutorial videos below. Learn more about.
  1. OWASP AppSec Basics: http://www.youtube.com/watch?v=CDbWvEwBBxo
  2. OWASP SQL Injection: http://www.youtube.com/watch?v=pypTYPaU7mM
  3. OWASP Cross Site Scripting: http://www.youtube.com/watch?v=_Z9RQSnf8-g
  4. OWASP Strict Transport Security: http://www.youtube.com/watch?v=zEV3HOuM_Vw

Tuesday 10 September 2013

Linux Containers on Virtualbox - Disposal Boxes by Michal Migurski's

Hey look, a month went by and I stopped blogging because I have a new job. Great.
One of my responsibilities is keeping an eye on our sprawling Github account, currently at 326 repositories and 151 members. The current fellows are working on a huge number of projects and I frequently need to be able to quickly install, test and run projects with a weirdly-large variety of backend and server technologies. So, it’s become incredibly important to me to be able to rapidly spin up disposable Linux web servers to test with. Seth clued me in to Linux Containers (LXC) for this:
LXC provides operating system-level virtualization not via a full blown virtual machine, but rather provides a virtual environment that has its own process and network space. LXC relies on the Linux kernel cgroups functionality that became available in version 2.6.24, developed as part of LXC. … It is used by Heroku to provide separation between their “dynos.”
I use a Mac, so I’m running these under Virtualbox. I move around between a number of different networks, so each server container had to have a no-hassle network connection. I’m also impatient, so I really needed to be able to clone these in seconds and have them ready to use.
This is a guide for creating an Ubuntu Linux virtual machine under Virtualbox to host individual containers with simple two-way network connectivity. You’ll be able to clone a container with a single command, and connect to it using a simple <container>.local host name.

The Linux Host

First, download an Ubuntu ISO. I try to stick to the long-term support releases, so I’m using Ubuntu 12.04 here. Get a copy of Virtualbox, also free.
Create a new Virtualbox virtual machine to boot from the Ubuntu installation ISO. For a root volume, I selected the VDI format with a size of 32GB. The disk image will expand as it’s allocated, so it won’t take up all that space right away. I manually created three partitions on the volume:
  1. 4.0 GB ext4 primary.
  2. 512 MB swap, matching RAM size. Could use more.
  3. All remaining space btrfs, mounted at /var/lib/lxc.
Btrfs (B-tree file system, pronounced “Butter F S”, “Butterfuss”, “Better F S”, or “B-tree F S") is a GPL-licensed experimental copy-on-write file system. It will allow our cloned containers to occupy only as much disk space as is changed, which will decrease the overall file size of the virtual machine.
During the OS installation process, you’ll need to select a host name. I used “ubuntu-demo” for this demonstration.

Host Linux Networking

Boot into Linux. I started by installing some basics, for me: git, vim, tcsh, screen, htop, and etckeeper.
Set up /etc/network/interfaces with two bridges for eth0 and eth1, both DHCP. Note that eth0 and eth1 must be commented-out, as in this sample part of my /etc/network/interfaces:
## The primary network interface
#auto eth0
#iface eth0 inet dhcp

auto br0
iface br0 inet dhcp
        dns-nameservers 8.8.8.8
        bridge_ports eth0
        bridge_fd 0
        bridge_maxwait 0

auto br1
iface br1 inet dhcp
        bridge_ports eth1
        bridge_fd 0
        bridge_maxwait 0
Back in Virtualbox preferencese, create a new network adapter and call it “vboxnet0”. My settings are 10.1.0.1, 255.255.255.0, with DHCP turned on.


Shut down the Linux host, and add the secondary interface in Virtual box. Choose host-only networking, the vboxnet0 adapter, and “Allow All” promiscuous mode so that the containers can see inbound network traffic.

The primary interface will be NAT by default, which will carry normal out-bound internet traffic.
  1. Adapter 1: NAT (default)
  2. Adapter 2: Host-Only vboxnet0
Start up the Linux host again, and you should now be able to ping the outside world.
% ping 8.8.8.8

PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_req=1 ttl=63 time=340 ms
…
Use ifconfig to find your Linux IP address (mine is 10.1.0.2), and try ssh’ing to that address from your Mac command line with the username you chose during initial Ubuntu installation.
% ifconfig br1

br1       Link encap:Ethernet  HWaddr 08:00:27:94:df:ed  
          inet addr:10.1.0.2  Bcast:10.1.0.255  Mask:255.255.255.0
          inet6 addr: …
Next, we’ll set up Avahi to broadcast host names so we don’t need to remember DHCP-assigned IP addresses. On the Linux host, install avahi-daemon:
% apt-get install avahi-daemon
In the configuration file /etc/avahi/avahi-daemon.conf, change these lines to clarify that our host names need only work on the second, host-only network adapter:
allow-interfaces=br1,eth1
deny-interfaces=br0,eth0,lxcbr0
Then restart Avahi.
% sudo service avahi-daemon restart
Now, you should be able to ping and ssh to ubuntu-demo.local from within the virtual machine and your Mac command line.

No Guest Containers

So far, we have a Linux virtual machine with a reliable two-way network connection that’s resilient to external network failures, available via a meaningful host name, and with a slightly funny disk setup. You could stop here, skipping the LXC steps and use Virtualbox’s built-in cloning functionality or something like Vagrant to set up fresh development environments. I’m going to keep going and set up LXC.

Linux Guest Containers

Install LXC.
% sudo apt-get lxc
Initial LXC setup uses templates, and on Ubuntu there are several useful ones that come with the package. You can find them under /usr/lib/lxc/templates; I have templates for ubuntu, fedora, debian, opensuse, and other popular Linux distributions. To create a new container called “base” use lxc-create with a chosen template.
% sudo lxc-create -n base -t ubuntu
This takes a few minutes, because it needs retrieve a bunch of packages for a minimal Ubuntu system. You’ll see this message at some point:
##
# The default user is 'ubuntu' with password 'ubuntu'!
# Use the 'sudo' command to run tasks as root in the container.
##
Without starting the container, modify its network adapters to match the two we set up earlier. Edit the top of /var/lib/lxc/base/config to look something like this:
lxc.network.type=veth
lxc.network.link=br0
lxc.network.flags=up
lxc.network.hwaddr = 00:16:3e:c2:9d:71

lxc.network.type=veth
lxc.network.link=br1
lxc.network.flags=up
lxc.network.hwaddr = 00:16:3e:c2:9d:72
An initial MAC address will be randomly generated for you under lxc.network.hwaddr, just make sure that the second one is different.
Modify the container’s network interfaces by editing /var/lib/lxc/base/rootfs/etc/network/interfaces (/var/lib/lxc/base/rootfs is the root filesystem of the new container) to look like this:
auto eth0
iface eth0 inet dhcp
        dns-nameservers 8.8.8.8

auto eth1
iface eth1 inet dhcp
Now your container knows about two network adapters, and they have been bridged to the Linux host OS virtual machine NAT and host-only adapters. Start your new container:
% sudo lxc-start -n base
You’ll see a normal Linux login screen at first, use the default username and password “ubuntu” and “ubuntu” from above. The system starts out with minimal packages. Install a few so you can get around, and include language-pack-en so you don’t get a bunch of annoying character set warnings:
% sudo apt-get install language-pack-en
% sudo apt-get install git vim tcsh screen htop etckeeper
% sudo apt-get install avahi-daemon
Make a similar change to the /etc/avahi/avahi-daemon.conf as above:
allow-interfaces=eth1
deny-interfaces=eth0
Shut down to return to the Linux host OS.
% sudo shutdown -h now
Now, restart the container with all the above modifications, in daemon mode.
% sudo lxc-start -d -n base
After it’s started up, you should be able to ping and ssh to base.local from your Linux host OS and your Mac.
% ssh ubuntu@base.local

Cloning a Container

Finally, we will clone the base container. If you’re curious about the effects of Btrfs, check the overall disk usage of the /var/lib/lxc volume where the containers are stored:
% df -h /var/lib/lxc

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda3        28G  572M   26G   3% /var/lib/lxc
Clone the base container to a new one, called “clone”.
% sudo lxc-clone  -o base -n clone
Look at the disk usage again, and you will see that it’s not grown by much.
% df -h /var/lib/lxc

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda3        28G  573M   26G   3% /var/lib/lxc
If you actually look at the disk usage of the individual container directories, you’ll see that Btrfs is allowing 1.1GB of files to live in just 573MB of space, representing the repeating base files between the two containers.
% sudo du -sch /var/lib/lxc/*

560M /var/lib/lxc/base
560M /var/lib/lxc/clone
1.1G total
You can now start the new clone container, connect to it and begin making changes.
% sudo lxc-start -d -n clone
% ssh ubuntu@clone.local

Conclusion

I have been using this setup for the past few weeks, currently with a half-dozen containers that I use for a variety of jobs: testing TileStache, installing Rails applications with RVM, serving Postgres data, and checking out new packages. One drawback that I have encountered is that as the disk image grows, my nightly time machine backups grow considerably. The Mac host OS can only see the Linux disk image as a single file.
On the other hand, having ready access to a variety of local Linux environments has been a boon to my ability to quickly try out ideas. Special thanks again to Seth for helping me work through some of the networking ugliness.

Further Reading

Tao of Mac has an article on a similar, but slightly different Virtualbox and LXC setup. They don’t include the promiscuous mode setting for the second network adapter, which I think is why they advise using Avahi and port forwarding to connect to the machine. I believe my way here might be easier.
Shift describes a Vagrant and LXC setup that skips Avahi and uses a plain hostnames for internal connectivity.

The Owner of this post is Michal Migurski
Find is Blog here http://mike.teczno.com/notes/disposable-virtualbox-lxc-environments.html 

Thursday 11 July 2013

Solve VMWare Workstation 9 Error on Linux Kernel 3.8.0-26



I have VMware Workstations 9 installed on my uBuntu 13 machine and today without any apparent reason it stop working throwing me the following error, every time I've tried to boot up any VM machine.


To solver the problem .. here is what I have done ... created bash script to handle problems with VMware Player on 13.04 ...




  • #!/bin/bash
  • if [[ $UID != 0 ]]; then
  •     echo "Please run this script with sudo:"
  •     echo "sudo $0 $*"
  •     exit 1
  • fi
  • sudo ln -s /usr/src/linux-headers-$(uname -r)/include/generated/uapi/linux/version.h /usr/src/linux-headers-$(uname -r)/include/linux/version.h
  • cd /usr/lib/vmware/modules/source
  • sudo tar -xf vmci.tar
  • cd vmci-only
  • sudo sed '127s/.*/   .remove = vmci_remove_device,/' driver.c > driver.c.tmp
  • mv driver.c.tmp driver.c
  • sudo sed '1753s/.*/static int/' driver.c > driver.c.tmp
  • mv driver.c.tmp driver.c
  • sudo sed '1981s/.*/static void/' driver.c > driver.c.tmp
  • mv driver.c.tmp driver.c
  • cd ..
  • sudo tar -cf vmci.tar vmci-only/
  • sudo rm vmci-only/ -Rf
  • sudo vmware-modconfig --console --install-all
  • sudo rm /usr/src
  • Before write the script I've open the terminal and type " #sudo kate " then when kate opened I entered the script and saved on " /usr/src/open-vm-tools-xxxx.xx.xx" (replace the x with year month and day) and save it. Then close Kate

    The Open terminal again and do the following;


    after you should have VMware workstation running...



    This is another posting showing "how to do computing" for everyday computer usages... For a general public

    Friday 21 June 2013

    Thursday 16 May 2013

    Update script working with Backtrack 5

    I could not find an update script working with Backtrack 5 so I decided to make a quick one.

    Updated version: 2.0

    Changelog Version 2.0
    Code:
    * Reviewed all tools.
    * Compatible with Backtrack 5 R2/R3.
    * Removed the menu.
    Code:
    git clone git://github.com/sickn3ss/backtrack_update.git
    cd backtrack_update
    chmod a+x backtrack5_update.py
    ./backtrack5_update.py
    Screenshot:



    If you know more tools I could add to the script in order to update them please feel free to post them here.
    Also if you find any bugs or have any feedback feel free to contact me in this thread. I will update the thread with each update to the script.

    NOTE: Please make sure you understand what the script does before running it.

    Enjoy!

    Monday 13 May 2013

    How to Upgrade Wine from the Older version to Wine1.5 on uBuntu

    I am new to Wine and anything like it. I want to run the application "ATCS Monitor". When I installed ATCS Monitor I received an error message:

    C:\windows\system32\wshom.ocx

    Unable to register the DLL/OCX: RegSvr32 failed with exit code 0x1

    -From Terminal-

    err:typelib:sltg_get_typelib_ref Unable to find reference
    err:module:import_dll Library ScrRun.dll (which is needed by L"C:\\windows\\system32\\wshom.ocx") not found


    I can get through it by just ignoring it but I then have errors in ATCS Monitor.

    First, upgrade to Wine 1.3.35. Wine 1.2.3 is almost a year old and is 
    missing some very important updates. 

    Second, do a search for the file in the directory $HOME/.wine I suspect 
    that either it is not there or it is in the same directory as where the 
    program installed. ...


    Ubuntu and Ubuntu derivatives (such as Kubuntu) obtain software from packages which are stored in repositories. Ubuntu's default repository includes Wine, however if you want to stay up to date with the latest Wine package you can use WineHQ's by following these instructions.

    Adding the WineHQ PPA Repository:

    Open the Software Sources menu by launching the Ubuntu Software Center and selecting Edit->Software Sources. Choose the Other Software tab and click Add.
    Software Center->Edit->Software Sources->Other Software
    Then, copy and paste the line below.
    ppa:ubuntu-wine/ppa

    Warning: Beta packages

    The 1.5 packages here are beta packages. This means they will periodically suffer from regressions, and as a result an update may break functionality in Wine. If the stable 1.4 Wine version works for you, then you may not want to use these beta packages.

    Installing Wine:

    Once you have added the WineHQ PPA Repository, you are ready to install.
    To get the most recent Wine 1.5 beta, click this link to install the wine1.5 package.
    To install the older, stable Wine 1.4 version, click this link to install the wine1.4 package.

    Upgrading to a new version of Ubuntu

    If you are upgrading the entire system, such as going from Ubuntu 11.10 to 12.04, you will need to come back to this page and add the repository again. The built in update manager will not switch the Wine repository automatically.

    Alternative Command Line Instructions for Installing Wine:

    It is also possible to add the Wine PPA and install via the terminal. This may be useful on Kubuntu, Xubuntu, and other Ubuntu derivatives.
    sudo add-apt-repository ppa:ubuntu-wine/ppa
    Then update APT package information by running 'sudo apt-get update'. You can now install Wine by typing 'sudo apt-get install wine1.5'.
    If you'd like to browse the PPA manually, you can visit its Launchpad page.

    Thursday 13 December 2012

    Using Virtual Ethernet Adapters in Promiscuous Mode on a Linux Host


    VMware Workstation does not allow the virtual Ethernet adapter to go into promiscuous mode unless the user running VMware Workstation has permission to make that setting. This follows the standard Linux practice that only root can put a network interface into promiscuous mode.

    When you install and configure VMware Workstation, you must run the installation as root. VMware Workstation creates the VMnet devices with root ownership and root group ownership, which means that only root has read and write permissions to the devices.

    To set the virtual machine's Ethernet adapter to promiscuous mode, you must launch VMware Workstation as root because you must have read and write access to the VMnet device. For example, if you are using bridged networking, you must have access to /dev/vmnet0.

    To grant selected other users read and write access to the VMnet device, you can create a new group, add the appropriate users to the group and grant that group read and write access to the appropriate device. You must make these changes on the host operating system as root (su -). For example, you can enter the following commands:

    chgrp <newgroup> /dev/vmnet0

    chmod g+rw /dev/vmnet0

    <newgroup> is the group that should have the ability to set vmnet0 to promiscuous mode.
    The command to run vmware workstations ads root is simple: user@user#:~$ sudo vmware start

    If you want all users to be able to set the virtual Ethernet adapter (/dev/vmnet0 in our example) to promiscuous mode, run the following command on the host operating system as root:

    chmod a+rw /dev/vmnet0


    This is another posting showing "how to do computing" for everyday computer usages... For a general public

    Mastering Docker Minified Systems: A Step-by-Step Guide with Real Use Cases

    Introduction Docker is a powerful platform for developing, shipping, and running applications. Minified Docker systems are optimized for siz...