Using ssh-agent in fish-shell

When we use zsh, we evaluate ssh-agent via this command.

1
$ eval `ssh-agent`

Unlike other shells, fish-shell does not use backticks ` for command substitutions.
Instead, it uses parentheses.

So I tried $ eval (ssh-agent), but errors occurred.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
$ eval (ssh-agent)
- (line 1): Unsupported use of '='. In fish, please use 'set SSH_AUTH_SOCK /var/folders/h_/yh5lh2hd7zv0r9qkncx49_bw0000gn/T//ssh-DyB4EGd5T2DZ/agent.43633'.
begin; SSH_AUTH_SOCK=/var/folders/h_/yh5lh2hd7zv0r9qkncx49_bw0000gn/T//ssh-DyB4EGd5T2DZ/agent.43633; export SSH_AUTH_SOCK; SSH_AGENT_PID=43634; export SSH_AGENT_PID; echo Agent pid 43634;
^
from sourcing file -
called on line 60 of file /usr/local/Cellar/fish/2.6.0/share/fish/functions/eval.fish
in function 'eval'
called on standard input
- (line 1): Unsupported use of '='. In fish, please use 'set SSH_AGENT_PID 43634'.
begin; SSH_AUTH_SOCK=/var/folders/h_/yh5lh2hd7zv0r9qkncx49_bw0000gn/T//ssh-DyB4EGd5T2DZ/agent.43633; export SSH_AUTH_SOCK; SSH_AGENT_PID=43634; export SSH_AGENT_PID; echo Agent pid 43634;
^
from sourcing file -
called on line 60 of file /usr/local/Cellar/fish/2.6.0/share/fish/functions/eval.fish
in function 'eval'
called on standard input
Agent pid 43634

It’s due to how variables are set.
To work around this, we have to use the csh-style option -c

1
$ eval (ssh-agent -c)

Works cited

Share Comments

Against the Meltdown and Spectre for RHEL

Now, a lot of web developers are making effort to against Meltdown and Spectre.

This is an article of TechCrunch about them.
Kernel panic! What are Meltdown and Spectre, the bugs affecting nearly every computer and device?

One of the most popular Linux distributions is Red-Hat-Enterprize-Linux(RHEL) including CentOS.
Maybe you use it.

RHEL mentions Spectre and Meltdown in their blog.
What are Meltdown and Spectre? Here’s what you need to know.

Also, it provides updated rpm-packages for affected products.
Kernel Side-Channel Attacks - CVE-2017-5754 CVE-2017-5753 CVE-2017-5715

If you want to take actions, visit above page and click ‘Resolve’ tab.
You can see some packages which should be updated.

For instance, if you use RHEL-7, it’s needed to update these packages.

  • kernel
  • kernel-rt
  • libvirt
  • qemu-kvm
  • dracut

They are provided via yum, so you can update them by this command.

1
$ sudo yum update kernel kernel-rt libvirt qemu-kvm dracut

Keep on collecting information about this problem.

Share Comments

Start using fish-shell and fisherman

There are plenty of command line shell.
I had used to use zsh but recently replaced it to fish.

What is fish-shell

Introduce official description.

fish is a smart and user-friendly command line shell for macOS, Linux, and the rest of the family. fish includes features like syntax highlighting, autosuggest-as-you-type, and fancy tab completions that just work, with no configuration required.

I introduced fish because of these points.

  • easy to install
  • powerful built-in functions such as autosuggestion
  • having good plugin manager(below)

Install fish-shell

For macOS, it can be installed with Homebrew.

1
$ brew install fish

For other linux distributions, many packages are available.
https://software.opensuse.org/download.html?project=shells%3Afish%3Arelease%3A2&package=fish

I also use CentOS 6, I added a yum repository and installed fish from it.
As a note, you have to be root.

1
2
3
# cd /etc/yum.repos.d/
# wget https://download.opensuse.org/repositories/shells:fish:release:2/CentOS_6/shells:fish:release:2.repo
# yum install fish

Install fisherman

fisherman is a plugin manager for fish-shell.
It lets us to install and remove fish-shell plugins easily.

To install itself, we use curl command.

1
curl -Lo ~/.config/fish/functions/fisher.fish --create-dirs https://git.io/fisher

To install plugins, we use fisher install command(or just fisher command).

1
$ fisher install fzf

Plugins I highly recommend

z

z remember history of changing directories.
It enables you to jump to the directory from everywhere.

Install it with fisherman.

1
$ fisher z

Here is official repository.

fzf

fif is a command-line fuzzy finder.
For instance, we can search command from history interactively.

To install for fish, it is prerequisite to install fzf in your system.

Using Homebrew

1
$ brew install fzf

Using git

1
2
$ git clone --depth 1 https://github.com/junegunn/fzf.git ~/.fzf
$ ~/.fzf/install

After that, install fzf with fisher.

1
$ fisher fzf

Here is official repository.

bobthefish(theme from oh-my-fish)

fisher can manage plugins for oh-my-fish.
Install bobthefish with omf/ namespace.

1
$ fisher omf/theme-bobthefish

You can watch demo in official repository.
https://github.com/oh-my-fish/theme-bobthefish#bobthefish

Share Comments

Put CloudWatch alarm from cli

I’m using AWS and managing some EC2 instances.
When some of them have got wrong and status check failed, I used to reboot them by myself.

I wanted my instances to be rebooted automatically when it happens, so I put reboot action to CloudWatch alarm.

First, described my StatusCheckFailed-alarm-settings for checking current configuration.

1
2
3
$ aws --profile <my-profile> --region <my-region> cloudwatch describe-alarms \
| jq -C '.MetricAlarms | map(select(.MetricName == "StatusCheckFailed"))' \
| less -R

Then, I listed EC2 instances with their instance-id and hostname.
I registered a tag to instance that’s key is “Name” and value is instance’s hostname.

1
2
$ aws --profile <my-profile> --region <my-region> ec2 describe-instances \
| jq '.Reservations[].Instances | map(.InstanceId, [.Tags[] | select(.Key == "Name") | .Value])'

I wanted to make a list of under format but I’m not familiar with jq command, so I had to process after it.

1
2
3
"<instance-id-a> <hostname-a>"
"<instance-id-b> <hostname-b>"
"<instance-id-c> <hostname-c>"

If you’re good at jq, please tell me your practice;)

I got a mappings of instance-id and hostname.
Then, I wrote a shell script to register CloudWatch alarms.
I set action of the alarm not only rebooting the instance but also notifying with email.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
#!/bin/bash -ex
INSTANCES=(
"<instance-id-a> <hostname-a>"
"<instance-id-b> <hostname-b>"
"<instance-id-c> <hostname-c>"
)
for i in "${INSTANCES[@]}"; do
INSTANCE_ID=$(echo $i | cut -d " " -f 1)
EC2_HOSTNAME=$(echo $i | cut -d " " -f 2)
aws --profile <my-profile> --region <my-region> cloudwatch put-metric-alarm \
--alarm-name "[${EC2_HOSTNAME}] Instance is down." \
--actions-enabled \
--alarm-actions arn:aws:swf:<my-region>:<my-customer-id>:action/actions/AWS_EC2.InstanceId.Reboot/1.0 \
<my-sns-topic-arn> \
--metric-name StatusCheckFailed \
--namespace AWS/EC2 \
--statistic Maximum \
--dimensions Name=InstanceId,Value=${INSTANCE_ID} \
--period 60 \
--evaluation-periods 2 \
--threshold 1 \
--comparison-operator GreaterThanOrEqualToThreshold
done

Finally, I fired it and all alarms are registered successfully;)

Share Comments

Nginx makes tmp files under /var/lib/nginx by default

I rebuilt Nginx because I needed the following two packages.

After installed it, it seemed work normally.
But when it received POST requests, errors occurred.

Here is an error log.
Nginx makes temporary files when POST request came.

1
2
3
4
<accessed-datetime> [warn] 28195#0: *8938023 a client request body is buffered to a temporary file
/var/lib/nginx/tmp/client_body/0001310410, client: <client-ip-address>, server: <my-wordpress-server.com>, request: "POST
/wp-admin/post.php HTTP/1.1", host: "<my-wordpress-host.com>", referrer: "https://<my-wordpress-site.com>/wp-admin/post.php?
post=3084&action=edit"

The user of Nginx worker processes is nginx by default.
But in this case, the user of them is different from the original.

1
2
3
4
5
6
# ps auxf | { head -1; grep nginx; }
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1153 0.0 1.6 144076 65284 ? Ss Apr21 0:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
<our-nginx-user> 28195 0.1 2.9 601260 118728 ? Sl May02 340:37 \_ nginx: worker process
<our-nginx-user> 28196 0.1 2.9 599368 118216 ? Sl May02 348:31 \_ nginx: worker process
<our-nginx-user> 28197 0.0 0.4 219904 16260 ? Sl May02 1:25 \_ nginx: cache manager process

So, I changed the owner of the directory recursively.

1
# chown -R <our-nginx-user> /var/lib/nginx

We have to think deeply who should run middleware.

Share Comments

Tweet latest hexo post from Node.js

I sometimes write blog posts and I want to announce it in twitter.

I opened this blog some in last month, but few people come ;_;
I asked to my friend who knows much about SEO, and he gave me advice that I should share my post in twitter.
Even you have no follower in twitter, It’s still good for SEO.

This blog is made by hexo so I searched plugins to do it, but I couldn’t.
So, I made Node.js scripts.

Create Node.js scripts which announce latest post in twitter

1. Create twitter application and get access tokens.

Before create app, you have to register your phone number to your twitter account.
Signin twitter and access here.
https://twitter.com/settings/devices

Then, you can create your own twitter application in here.
https://apps.twitter.com/app/new

After create, click ‘Keys and Access Tokens’ tab.
You can get api keys and access tokens from it!

2. Install helpful npm plugins

Start writing Node.js codes ;)

I used those packages.

  • dotenv - zero-dependency module that loads environment variables from a .env file into process.env
  • cheerio - Fast, flexible & lean implementation of core jQuery designed specifically for the server.
  • twitter - An asynchronous client library for the Twitter REST and Streaming API’s.

Install packages.

1
$ npm install dotenv cheerio twitter --save

3. Define twitter api keys in .env file

I wondered how manage my secret information in Node.js.
Finally I decided that I write them in .env and read it from other files.

$ vim .env

1
2
3
4
TWITTER_API_CONSUMER_KEY=xxxxxxx
TWITTER_API_CONSUMER_SECRET=xxxxxxx
TWITTER_API_ACCESS_TOKEN_KEY=xxxxxxx
TWITTER_API_ACCESS_TOKEN_SECRET=xxxxxxx

This is secret information so I make git to ignore it.
$ vim .gitignore

1
+.env

I wrote a file that have config data.
$ vim secrets.js

1
2
3
4
5
6
7
8
9
10
11
12
require('dotenv').config()
module.exports = {
twitterApi: {
keysAndAccessTokens: {
consumer_key: process.env.TWITTER_API_CONSUMER_KEY,
consumer_secret: process.env.TWITTER_API_CONSUMER_SECRET,
access_token_key: process.env.TWITTER_API_ACCESS_TOKEN_KEY,
access_token_secret: process.env.TWITTER_API_ACCESS_TOKEN_SECRET,
},
},
}

4. Write scripts posts to twitter.

At first, this parses site.xml and extract the latest post’s url.
Then it posts it to twitter!
$ vim bin/auto_tweet.js

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
const fs = require('fs'),
$ = require('cheerio'),
Twitter = require('twitter');
const twitterApiConfig = require('../secrets').twitterApi;
const client = new Twitter(twitterApiConfig.keysAndAccessTokens);
fs.readFile('.deploy_git/sitemap.xml', 'utf8', (err, xml) => {
if (err) {
throw err;
}
const latestPostUrl = $('loc', 'url', xml).first().text();
const params = {
status: latestPostUrl,
};
// Post tweet for notification.
client.post('statuses/update', params)
.then(tweet => {
console.log(tweet);
})
.catch(error => {
throw error;
});
});

5. Execute the scripts

1
$ node bin/auto_tweet.js

Here is my whole codes ;)
https://github.com/alpaca0984/alpaca-tech-farm/pull/15/files

Share Comments

Remove old docker images from ECR and docker-build-server

1. Remove old images from ECR

Amazon ECR registries is limited to numbers of images we can save in it.
We have to delete old images and I wrote a script to realize it from command-line.

I use aws-cli.
Here is official document of $ aws ecr desribe-images command.
http://docs.aws.amazon.com/cli/latest/reference/ecr/describe-images.html

1
2
3
4
5
6
KEEP_IMAGES=2 # number of images you want to remain
aws ecr describe-images --repository-name <your-repository-name> --query 'imageDetails[]' \
| jq --raw-output 'sort_by(.imagePushedAt) | reverse | .[].imageDigest' \
| awk "NR > ${KEEP_IMAGES}" \
| xargs -I{} aws ecr batch-delete-image --repository-name <your-repository-name> --image-ids imageDigest={}

2. Remove old images from docker building server

Remove old deployed images

This page teaches us how to use format options of $ docker image ls command
https://docs.docker.com/engine/reference/commandline/images/#format-the-output

1
2
3
4
5
6
7
8
9
10
11
KEEP_IMAGES=1 # number of images you want to remain
# Before you remove images, you have to remove tags from it.
docker image ls --filter 'reference=<tag-pattern>' --format '{{.Repository}}:{{.Tag}}' \
| awk "NR > ${KEEP_IMAGES}" \
| xargs docker image rm
# Remove images.
docker image ls --quiet --filter 'reference=<pattern>' \
| awk "NR > ${KEEP_IMAGES}" \
| xargs docker image rm

Delete images which are created when you fail to build image

When we failed to build image, <none>:<none> image might be created.
We can pick them up from $ docker image ls command with filtering 'dangling=true'.

1
2
3
4
5
6
7
8
# Before you remove image, you have to remove active containers.
docker image ls --quiet --filter 'dangling=true' \
| xargs -I{} docker container ls --all --quiet --filter 'ancestor={}' \
| xargs docker container rm
# Remove images.
docker image ls --quiet --filter "dangling=true" \
| xargs docker image rm
Share Comments

Call Azure's Text Analytics API from JavaScript

Previously, I created docker environment with Nginx, Rails and Postgres.
Nginx, Rails and Postgres with docker-compose

In it, I ran a rails application with Azure’s Text Analytics API.
With it, I made app which detect key phrases from text.

Register Azure and get trial access token of the API

You can do it from here (free trial token valids only 30 days).
Microsoft Azure - Text Analytics API

Setup in rails

Create controller in rails container.
I created docker environment in this post.

1
$ docker-compose run app bundle exec rails g controller smash

Configure routing.
$ vim config/routes.rb

1
2
3
4
5
Rails.application.routes.draw do
# For details on the DSL available within this file, see http://guides.rubyonrails.org/routing.html
+ resources :smash, only: :index
+ root 'smash#index'
end

Write view and call the API

This is over view.

  1. Set text to a textare.
  2. Press submit button.
  3. We send the text to Text Analytics API and receive extracted words.

Here is my code.
In the code calling API, you have to pass your API key which you get in your Azure member page.
$ vim app/views/smash/index.erb

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
<!-- Cute Alpaca icon and title. -->
<div style="margin-bottom: 16px">
<img style="width: 38px; height: 38px" src="https://i.pinimg.com/originals/a6/bd/07/a6bd07140b8e12855ce842f189d8fa36.jpg" alt="Smiley face" />
<span style="font-size: 32px">Smash text into words</span>
</div>
<!-- Input text area and submit button. -->
<div style="display: inline-block">
<textarea name="name" id="js-inputText" rows="8" cols="80"></textarea>
<button type="submit" name="submit" id="js-submit">Send</button>
</div>
<!-- Set results from calling API. -->
<p>Result</p>
<ul id="resultList" style="list-style: none"></ul>
<script type="text/javascript">
'use strict';
document.addEventListener("DOMContentLoaded", () => {
// When sugmit button clicked, call Azure Text Analytics API.
document.getElementById('js-submit').addEventListener('click', listener => {
const inputText = document.getElementById('js-inputText').value;
const requestBody = {
"documents": [
{
"language": "en",
"id": "1",
"text": inputText,
}
]
};
// API request
(requestBody => {
const request = new XMLHttpRequest();
request.open('POST', 'https://westus.api.cognitive.microsoft.com/text/analytics/v2.0/keyPhrases', true);
request.setRequestHeader("Content-Type", "application/json");
request.setRequestHeader("Ocp-Apim-Subscription-Key", "xxxxxxxxxxxxxxxxxxxxx"); // Set API key.
request.setRequestHeader("Accept","application/json");
request.onload = () => {
const resultList = document.getElementById('resultList');
// Clear resultList field.
resultList.innerHTML = '';
// Set response data.
JSON.parse(request.responseText).documents.forEach(result => {
result.keyPhrases.forEach(phrase => {
const li = document.createElement('li');
li.innerHTML = `<label><input type="checkbox" />${phrase}</label>`;
resultList.appendChild(li);
});
});
};
request.onerror = () => {
alert(request.responseText);
};
request.send(JSON.stringify(requestBody));
})(requestBody);
});
});
</script>

Here is my whole code of this including docker files;)
https://github.com/alpaca0984/text-smasher

Share Comments

Nginx, Rails and Postgres with docker-compose

I developed a web application environment with docker.
It includes three containers, Nginx, Rails(puma) and Postgres.

In addition, I created and ran a small application in it.
It smash text into words so I named it “text-smasher”.

In this page, I show you the recipes of my docker containers.

Create Rails Docker file

At first, I’m gonna create rails container and access it directly.

Make project directory.

1
$ mkdir text-smasher && cd $_

Make directories where I’m gonna put Docker file in.

1
$ mkdir -p docker/app

Create docker file for rails.
$ vim docker/app/Dockerfile

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# Base image:
FROM ruby:2.3.3
# Install dependencies
RUN apt-get update -qq && apt-get install -y build-essential libpq-dev nodejs
# create application directory
RUN mkdir /myapp
# Set our working directory inside the image
WORKDIR /myapp
ADD Gemfile /myapp/Gemfile
ADD Gemfile.lock /myapp/Gemfile.lock
RUN bundle install
ADD . /myapp
EXPOSE 3000
CMD [ "bundle", "exec", "puma", "-C", "config/puma.rb" ]

Create docker-compose file

$ vim docker-compose.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
version: '3'
services:
app:
build:
context: .
dockerfile: ./docker/app/Dockerfile
volumes:
- .:/myapp
depends_on:
- db
ports:
- 3000:3000
db:
image: postgres

Init Rails Project

Create Gemfile.

1
$ bundle init

Activate rails gem.
$ vim Gemfile

1
2
3
4
5
6
source "https://rubygems.org"
git_source(:github) {|repo_name| "https://github.com/#{repo_name}" }
-# gem "rails"
+gem "rails"

Install gems and create rails project.
You may be asked if you wanna overwrite gemfile, then press ‘Y’.

1
$ docker-compose run app bundle exec rails new . -d postgresql

Set db host to postgres container.
This is just sample, and you shoud define properly username and password.
$ vim config/database.yml

1
2
3
4
5
6
7
8
default: &default
adapter: postgresql
encoding: unicode
+ host: db
+ username: postgres
# For details on connection pooling, see Rails configuration guide
# http://guides.rubyonrails.org/configuring.html#database-pooling
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>

Build docker image and create database.

1
2
$ docker-compose build
$ docker-compose run app bundle exec rails db:create RAILS_ENV=production

Run application.

1
$ docker-compose up

All containers work well.

1
2
3
4
5
$ docker-compose ps
Name Command State Ports
--------------------------------------------------------------------------------
textsmasher_app_1 bundle exec puma -C config ... Up 0.0.0.0:3000->3000/tcp
textsmasher_db_1 docker-entrypoint.sh postgres Up 5432/tcp

I could see Rails’s welcome page through localhost:3000 !

Add Nginx’s container

Second, I’m gonna create Nginx’s container and let it proxy to Puma.
I used this page as reference.
Codepany Blog > RAILS 5 AND DOCKER (PUMA, NGINX)

Create a directory which is related to nginx container.

1
$ mkdir docker/web

Add nginx config file to proxy rails application.
$ vim docker/web/app.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
upstream puma_rails_app {
server app:3000;
}
server {
listen 80;
proxy_buffers 64 16k;
proxy_max_temp_file_size 1024m;
proxy_connect_timeout 5s;
proxy_send_timeout 10s;
proxy_read_timeout 10s;
location / {
try_files $uri $uri/ @rails_app;
}
location @rails_app {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://puma_rails_app;
# limit_req zone=one;
access_log /var/www/text-smasher/log/nginx.access.log;
error_log /var/www/text-smasher/log/nginx.error.log;
}
}

Create docker file for nginx.
It refers config file I created in last step.
$ vim docker/web/Dockerfile

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
# Base image:
FROM nginx
# Install dependencies
RUN apt-get update -qq && apt-get -y install apache2-utils
# establish where Nginx should look for files
ENV RAILS_ROOT /var/www/text-smasher
# Set our working directory inside the image
WORKDIR $RAILS_ROOT
# create log directory
RUN mkdir log
# copy over static assets
COPY public public/
# Copy Nginx config template
COPY docker/web/app.conf /tmp/docker_example.nginx
# substitute variable references in the Nginx config template for real values from the environment
# put the final config in its place
RUN envsubst '$RAILS_ROOT' < /tmp/docker_example.nginx > /etc/nginx/conf.d/default.conf
EXPOSE 80
# Use the "exec" form of CMD so Nginx shuts down gracefully on SIGTERM (i.e. `docker stop`)
CMD [ "nginx", "-g", "daemon off;" ]

Change docker-compose file to create nginx container.
I’m gonna access rails application via nginx, so I deleted port mapping of it.
$ vim docker-compose.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
version: '3'
services:
+ web:
+ build:
+ context: .
+ dockerfile: ./docker/web/Dockerfile
+ depends_on:
+ - app
+ ports:
+ - 8080:80
app:
build:
context: .
dockerfile: ./docker/app/Dockerfile
volumes:
- .:/myapp
depends_on:
- db
- ports:
- - 3000:3000
db:
image: postgres

Build and run the docker image

1
2
$ docker-compose build
$ docker-compose up

All containers work well.

1
2
3
4
5
6
$ docker-compose ps
Name Command State Ports
------------------------------------------------------------------------------
textsmasher_app_1 bundle exec puma -C config ... Up 3000/tcp
textsmasher_db_1 docker-entrypoint.sh postgres Up 5432/tcp
textsmasher_web_1 nginx -g daemon off; Up 0.0.0.0:8080->80/tcp

When I access to localhost:8080, I can see rails’s welcome page again!

I will write another post about the content of “text-smasher”.
It is coming soon;)
I wrote it!
Call Azure's Text Analytics API from JavaScript

Share Comments

Create yum repository

Previously, I created vim-rpm.
Create vim rpm

Next, I’m gonna create my own yum repository which installs above rpms with dependencies.

Prerequisites

Install a tool for creating repositories.

1
$ sudo yum install createrepo

Init repository and add packages

Create directory and init repository.

1
2
$ mkdir yumrepos && cd $_
$ createrepo .

Setted up repository data.

1
2
3
4
5
6
7
8
9
10
11
[vagrant@localhost yumrepos]$ tree repodata/
repodata/
├── 01a3b489a465bcac22a43492163df43451dc6ce47d27f66de289756b91635523-filelists.sqlite.bz2
├── 401dc19bda88c82c403423fb835844d64345f7e95f5b9835888189c03834cc93-filelists.xml.gz
├── 5dc1e6e73c84803f059bb3065e684e56adfc289a7e398946574d79dac6643945-primary.sqlite.bz2
├── 6bf9672d0862e8ef8b8ff05a2fd0208a922b1f5978e6589d87944c88259cb670-other.xml.gz
├── 7c36572015e075add2b38b900837bcdbb8a504130ddff49b2351a7fc0affa3d4-other.sqlite.bz2
├── dabe2ce5481d23de1f4f52bdcfee0f9af98316c9e0de2ce8123adeefa0dd08b9-primary.xml.gz
└── repomd.xml
0 directories, 7 files

Create repository.
Here, I named one alpaca-main.

1
[vagrant@localhost yumrepos]$ mkdir -p alpaca-main/Packages

Add packages above directory.
In my case, I used vim packages created previous post.

1
2
3
4
5
6
7
8
9
10
[vagrant@localhost yumrepos]$ ls -la alpaca-main/Packages/
total 14880
drwxrwxr-x. 2 vagrant vagrant 4096 Aug 30 06:31 .
drwxrwxr-x. 3 vagrant vagrant 22 Aug 30 06:30 ..
-rw-rw-r--. 1 vagrant vagrant 6211132 Aug 30 06:31 vim-common-7.4.160-1.el7_3.alpaca.1.x86_64.rpm
-rw-rw-r--. 1 vagrant vagrant 6233660 Aug 30 06:31 vim-debuginfo-7.4.160-1.el7_3.alpaca.1.x86_64.rpm
-rw-rw-r--. 1 vagrant vagrant 1093852 Aug 30 06:31 vim-enhanced-7.4.160-1.el7_3.alpaca.1.x86_64.rpm
-rw-rw-r--. 1 vagrant vagrant 8788 Aug 30 06:31 vim-filesystem-7.4.160-1.el7_3.alpaca.1.x86_64.rpm
-rw-rw-r--. 1 vagrant vagrant 445508 Aug 30 06:31 vim-minimal-7.4.160-1.el7_3.alpaca.1.x86_64.rpm
-rw-rw-r--. 1 vagrant vagrant 1226428 Aug 30 06:31 vim-X11-7.4.160-1.el7_3.alpaca.1.x86_64.rpm

Write .repo to access rpms via yum command.
$ vim /etc/yum.repos.d/alpaca-main.repo

1
2
3
4
[alpaca-main]
name=alpaca-main.$releasever
baseurl=file:///home/vagrant/yumrepos
gpgcheck=0

Try install vim from my repository(disable built-in repositories of CentOS7).

1
$ sudo yum --disablerepo=updates,base --enablerepo=alpaca-main install vim

If you couldn’t find your packeages, try to clear cache.

1
$ sudo yum clean --enablerepo=<your-repo> all

I could install vim package from my alpaca-main repository!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
[vagrant@localhost yumrepos]$ sudo yum --disablerepo=updates,base --enablerepo=alpaca-main install vim
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* extras: ftp.riken.jp
Resolving Dependencies
--> Running transaction check
---> Package vim-enhanced.x86_64 2:7.4.160-1.el7_3.alpaca.1 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=========================================================================================================================================
Package Arch Version Repository Size
=========================================================================================================================================
Installing:
vim-enhanced x86_64 2:7.4.160-1.el7_3.alpaca.1 alpaca-main 1.0 M
Transaction Summary
=========================================================================================================================================
Install 1 Package
Total download size: 1.0 M
Installed size: 2.2 M
Is this ok [y/d/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : 2:vim-enhanced-7.4.160-1.el7_3.alpaca.1.x86_64 1/1
Verifying : 2:vim-enhanced-7.4.160-1.el7_3.alpaca.1.x86_64 1/1
Installed:
vim-enhanced.x86_64 2:7.4.160-1.el7_3.alpaca.1
Complete!

Upload your packages wherever you want

I uploaded to Github.
https://github.com/alpaca0984/rpm-pkgs

And chage .repo file.
$ vim /etc/yum.repos.d/alpaca-main.repo

1
2
- baseurl=file:///home/vagrant/yumrepos
+ baseurl=https://raw.githubusercontent.com/alpaca0984/rpm-pkgs/master/x86_64

Create rpm for installing repository via yum

I don’t wanna put /etc/yum.repos.d/alpaca-main.repo every time.
So I’m gonna create rpm package which deploys above file.

Configure .rpmmacros.

$ vim ~/.rpmmacros

1
2
- %_topdir %(echo $HOME)/rpmbuild
+ %_topdir %(echo $HOME)/<your-repo-name>

Here, I named my package ‘alpaca-yum-repos’.

Set up build-tree.

1
$ rpmdev-setuptree

Then, rpm directories are prepared.

1
2
3
4
5
6
7
8
9
[vagrant@localhost ~]$ tree -d alpaca-yum-repos/
├── BUILD
├── repodata
├── RPMS
├── SOURCES
├── SPECS
└── SRPMS
6 directories

Add .repo file to SOURCE directory

Put alpaca-main.repo in SOURCE dir.
It’s used in .spec file.

1
$ cp /etc/yum.repos.d/alpaca-main.repo ~/alpaca-yum-repos/SOURCES/

Initialize spec file and update it

1
$ rpmdev-newspec SPECS/alpaca-yum-repos.spec

I got a template of spec file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
[vagrant@localhost alpaca-main]$ cat SPECS/alpaca-main.spec
Name: alpaca-main
Version:
Release: 1%{?dist}
Summary:
License:
URL:
Source0:
BuildRequires:
Requires:
%description
%prep
%setup -q
%build
%configure
make %{?_smp_mflags}
%install
rm -rf $RPM_BUILD_ROOT
%make_install
%files
%doc
%changelog

Update .spec file

$ vim SPECS/alpaca-yum-repos.spec

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
Name: alpaca-yum-repos
Version: 1
Release: 1%{?dist}
Summary: Alpaca0984's Original Packages for CentOS
Group: System Environment/Base
License: GPLv2
URL: https://github.com/alpaca0984/yum-repos
Source0: alpaca-main.repo
BuildArch: noarch
Requires: redhat-release >= %{version}
%description
%prep
%setup -q -c -T
%build
%install
rm -rf $RPM_BUILD_ROOT
# yum
install -dm 755 $RPM_BUILD_ROOT%{_sysconfdir}/yum.repos.d
install -pm 644 %{SOURCE0} \
$RPM_BUILD_ROOT%{_sysconfdir}/yum.repos.d
%clean
rm -rf $RPM_BUILD_ROOT
%files
%defattr(-,root,root,-)
%doc
%config(noreplace) /etc/yum.repos.d/*
%changelog

Build rpm package

Build package.

1
$ rpmbuild -ba SPECS/alpaca-yum-repos.spec

I got the rpm package!

1
2
3
4
5
6
[vagrant@localhost alpaca-main]$ tree RPMS
RPMS
└── noarch
└── alpaca-yum-repos-1-1.el7.centos.noarch.rpm
1 directory, 1 file

Install repository via yum

Finally, I got to install my alpaca-main repository.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
[vagrant@localhost alpaca-main]$ sudo yum install RPMS/noarch/alpaca-yum-repos-1-1.el7.centos.noarch.rpm
Loaded plugins: fastestmirror
Examining RPMS/noarch/alpaca-yum-repos-1-1.el7.centos.noarch.rpm: alpaca-yum-repos-1-1.el7.centos.noarch
Marking RPMS/noarch/alpaca-yum-repos-1-1.el7.centos.noarch.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package alpaca-yum-repos.noarch 0:1-1.el7.centos will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=========================================================================================================================================
Package Arch Version Repository Size
=========================================================================================================================================
Installing:
alpaca-yum-repos noarch 1-1.el7.centos /alpaca-yum-repos-1-1.el7.centos.noarch 130
Transaction Summary
=========================================================================================================================================
Install 1 Package
Total size: 130
Installed size: 130
Is this ok [y/d/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : alpaca-yum-repos-1-1.el7.centos.noarch 1/1
Verifying : alpaca-yum-repos-1-1.el7.centos.noarch 1/1
Installed:
alpaca-yum-repos.noarch 0:1-1.el7.centos
Complete!

As described above, I uploaded my rpm packages on Github!
You can install my alpaca repository.
https://github.com/alpaca0984/rpm-pkgs

Share Comments