Skip to content

YaCy Search Engine

Introduction

This describes on how to install a YaCy search engine.

Optional behind an HTTPS Apache proxy.

Installation

Create an LXC-container

Presuming LXC is already installed.

Create container type from selection out of list which is downloaded.

lxc-create -n search -t download

For this example ubuntu, jammy, amd64 is selected.

Start the container.

lxc-start -n search

Make the LXC-container run at start by changing or adding the following to the file /var/lib/lxc/search/config.

# Enable auto start of this container.
lxc.start.auto = 1
# Delay start this container 45 seconds from when the host starts.
lxc.start.delay = 45

Use a Docker Image

The available Docker image is yacy/yacy_search_server:latest.
The next shell script runs this image ands opens the related website.

#!/bin/bash

# Get this script working directory.
script_dir="$(realpath "$(dirname "${BASH_SOURCE[0]}")")"
# Name of the container.
container_name="yacy_search_server"
docker_image="yacy/yacy_search_server:latest"

# Show help when first argument is empty.
if [ -z "$1" ]; then
    echo "Usage: ${0} {run | shell | attach | stop | open}"
    exit 1
fi

# Check to check if a shell is to be created for inspecting the container content.
if [[ "${1}" == "shell" ]]; then
  # Create the mount directory.
  mkdir --parents "${script_dir}/data"
    # Remove the named container.
    docker rm -f "${container_name}"
    # Run a self named container mounting setting the config file location and mounting out structure.
    docker run \
    --name "${container_name}" \
    --publish 8090:8090 \
    --publish 8443:8443 \
    --volume "${script_dir}/data:/opt/yacy_search_server/DATA" \
    --log-opt max-size=200m \
    --log-opt max-file=2 \
    --interactive \
    --tty \
    "${docker_image}" \
    bash -l
fi

#--mount type=bind,source=$(pwd),target=/app,readonly \

# Check to check if a shell is to be created to run as daemon.
if [[ "${1}" == "run" ]]; then
  # Create the mount directory.
  mkdir --parents "${script_dir}/data"
    # Remove the named container.
    docker rm -f "${container_name}"
    # Run a self named container mounting setting the config file location and mounting out structure.
    docker run \
    --name "${container_name}" \
    --detach \
    --restart unless-stopped \
    --publish 8090:8090 \
    --publish 8443:8443 \
    --volume "${script_dir}/data:/opt/yacy_search_server/DATA" \
    --log-opt max-size=200m \
    --log-opt max-file=2 \
    "${docker_image}"
fi

# Check to check the container has to stop.
if [[ "${1}" == "stop" ]]; then
    docker stop "${container_name}"
fi 

# Check to check the container has to stop.
if [[ "${1}" == "open" ]]; then
    xdg-open "http://localhost:8090"
fi

# Check to check the container has to stop.
if [[ "${1}" == "attach" ]]; then
    docker exec -it "${container_name}" /bin/bash
fi

Container Configuration

To get hold of the containers console (root) execute the following.

lxc-attach -n search

Install the needed packages for easy use of commend line (bash-completion) and an editor (joe) or any other.

apt install bash-completion mc joe wget default-jre-headless

Make the shell use colors by modifying the .bashrc and uncomment the line #force_color_prompt=yes and also at the end of the same file uncomment the lines.

if [ -f /etc/bash_completion ] && ! shopt -oq posix; then
    . /etc/bash_completion
fi

Download the YaCy Tarball

Download the latest from here and (as of writing this is yacy_v1.924_20210209_10069.tar.gz) in to the temporary directory.

wget -O /tmp/yacy.tar.gz https://download.yacy.net/yacy_v1.924_20210209_10069.tar.gz

Extract the tarball into the /home/ubuntu directory with the command below resulting in a directory /home/ubuntu/yacy. User ubuntu is the default LXC container user for Ubuntu distro.

Extract

tar --extract --file /tmp/yacy.tar.gz --directory /home/ubuntu

Make sure the files are root user owned.

chown -R ubuntu:ubuntu /home/ubuntu/yacy

Create a Service

Calling /home/ubuntu/yacy/startYACY.sh seems not to work so the content of the bash script is used instead which works just fine.

Put the content below in a new service file named /etc/systemd/system/yacy.service. Notice that the user this service is running under is ubuntu.

[Unit]
Description=Yacy Crawler
After=network.target

[Service]
Type=simple
User=ubuntu
WorkingDirectory=/home/ubuntu/yacy
ExecStart=/home/ubuntu/yacy/startYACY.sh --foreground
ExecStop=/home/ubuntu/yacy/stopYACY.sh
Restart=always
Environment=
PIDFile=/home/ubuntu/yacy/DATA/yacy.running
TimeoutSec=120

[Install]
WantedBy=multi-user.target

Make systemd notice this new file.

systemctl daemon-reload

Check the yacy service status and start it and check again.

systemctl status yacy
systemctl status start
systemctl status yacy

Set Admin Credentials

As ubuntu user set the admin password.

su - ubuntu
~/yacy/bin/passwd.sh "my-secret-passwd"

Web Configuration

Open browser on the LXC-container which is usually <container-ip>:8090.
Go to the Administration pages using the Administration button at the top-left position.

Private Search Engine

To make this search engine be stand-alone and index only specific configured websites.

Section: 'First Steps'

Goto page 'Use Case & Account'.

Set the 'Use Case' to 'Search portal for your own web pages'.
Set the 'Peer Name' to a something sensible like 'Scanframe' for example.
The 'Peer Port' is by default 8090 and since it is set behind an HTTPS proxy server ssl and the router configuration through uPNP is not needed.

Section: Search Portal Integration

Goto page 'Portal Configuration'.

Set some of the fields like in Scanframe's case:

Field Content
Greeting Line Search Scanframe...
URL of Home Page https://search.scanframe.com
URL of a Small Corporate Image env/grafics/scanframe-small.png
URL of a Large Corporate Image env/grafics/scanframe-256x256.png
Alternative text for Corporate Images Scanframe

The env/ relative subdirectory is located actually directory /home/ubuntu/yacy/htroot/env.