Create your own S3 Object Storage!

Ever wonder how we can create own beautiful an object storage service like S3? If so, you’re in for a treat, because I’m about to show you how!

As we all know, bucket storage uses hash-based lookups for individual objects - which means operations like insertion, deletion, and lookup in a well-implemented object storage system have a time complexity of O(1). This is because a good hash function distributes elements evenly, allowing direct access to an object’s location.

Unlike a typical file manager that relies on nested folder structures, object storage uses fake nested folders — essentially just a naming convention using keys. Object storage systems also expose APIs for inserting, updating, deleting, and reading data. Services like Amazon S3 leverage these hash-based lookups, making them unbeatable in performance and scalability.

On top of that, when deployed in the cloud, services like S3 offer high scalability, availability, encryption (both at rest and in transit), fine-grained access control through IAM policies, and many other advanced features — all without compromising performance.

Amazon was the first to pull this off — they introduced the S3 service in 2006 with the pay-as-you-go model. It’s now no wonder that S3 became so popular and the default choice for most people. But what if I told you that you can achieve something similar using an open-source project? You’d probably think I’m joking, right? Well, I’m not — you heard that right.

in this article we will be using open-source project callled MinIO .

MinIO is an open-source, high-performance, S3-compatible object storage system. It allows you to store unstructured data (like image, photos, videos, backups, logs, etc.) just like AWS S3, but you can run it on your own infrastructure — whether that’s on-premise, in containers, or in the cloud. minio built on golang. that you can deploy anywhere — on your local server, Kubernetes cluster, or public cloud VM.

Key Features

• S3 API Compatible – Works seamlessly with AWS S3 SDKs and tools.

• Scalable – Can scale from a single node to distributed clusters with petabytes of data.

• Fast and Lightweight – Optimized for high-performance workloads like AI/ML, data lakes, and analytics.

• Secure – Supports TLS encryption, identity management, and fine-grained access control.

• Self-hosted – You control where and how your data is stored.

• **replication -**this feature ensures that your objects (files) are automatically copied (replicated) between different MinIO deployments or buckets — providing high availability, data durability, and disaster recovery.

• Storage classes - it offers a range of storage classes based on how often and how quickly files need to be accessed.

My Experience with MinIO

It’s been two years since we started using this solution in our organization, and we haven’t faced any problems so far. We had promised hardware and an object storage requirement, which led us to this solution. We began using it, and for our existing data, we used an S3 batch job to migrate data from S3 to MinIO. We are also using MinIO replication for object data backup.

Reasons you’d choose MinIO over a managed AWS S3

Self-Hosted & Complete Data Control

• You have full control over where and how your data is stored.

• Ideal for on-premises, private cloud, or hybrid deployments where compliance, security, or latency require local storage.

• You can avoid vendor lock-in and maintain data sovereignty.

Cost Efficiency

• MinIO has no egress fees, unlike AWS/GCP/Azure.

• You only pay for your own hardware and network, not per-request or per-GB costs.

• Excellent for large-scale workloads like analytics, AI/ML pipelines, and media repositories where cloud costs can explode.

Kubernetes & DevOps Friendly

• Designed for Kubernetes-native deployments using MinIO Operator and Helm charts.

• Integrates seamlessly with CI/CD pipelines and cloud-native apps.

Or maybe you’re just curious. You want to know how services like AWS S3, and how you could build one yourself.

No matter whevever reasons are, I’ve got you covered.

You might be thinking, “That sounds like a lot of work — hosting, deploying, and maintaining servers, right?”
Well, not really. With MinIO, you’re all set — most of the heavy lifting is already taken care of. And if you’re wondering how to do it, that’s exactly what I’m here to help with.

In this article, I’ll walk you through setting up a MinIO cluster on a Kubernetes environment. Using this approach, you can achieve the same level of availability and scalability through a container**-**based architecture. Of course, you can choose your own setup based on your preferences, but we’ll focus on deploying MinIO via Kubernetes.

main-course!

Prerequisites

• Kubernetes cluster ( i am running my k8s cluster on docker desktop on mac with one node)

• kubectl installed and configured

Creating things

---
apiVersion: v1
kind: Namespace
metadata:
  name: minio

---
apiVersion: v1
kind: Secret
metadata:
  name: minio-secret
  namespace: minio
type: Opaque
stringData:
  rootUser: minioadmin
  rootPassword: minioadmin123

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: minio-pv-0
  namespace: minio
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage
  hostPath:
    path: /tmp/minio-data
    type: DirectoryOrCreate

---
apiVersion: v1
kind: Service
metadata:
  name: minio-headless
  namespace: minio
spec:
  clusterIP: None
  ports:
    - port: 9000
      name: api
    - port: 9001
      name: console
  selector:
    app: minio

---
apiVersion: v1
kind: Service
metadata:
  name: minio
  namespace: minio
spec:
  type: LoadBalancer
  ports:
    - port: 9000
      targetPort: 9000
      name: api
    - port: 9001
      targetPort: 9001
      name: console
  selector:
    app: minio

---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: minio
  namespace: minio
spec:
  serviceName: minio-headless
  replicas: 1
  selector:
    matchLabels:
      app: minio
  template:
    metadata:
      labels:
        app: minio
    spec:
      containers:
      - name: minio
        image: minio/minio:latest
        args:
        - server
        - /data
        - --console-address
        - ":9001"
        env:
        - name: MINIO_ROOT_USER
          valueFrom:
            secretKeyRef:
              name: minio-secret
              key: rootUser
        - name: MINIO_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: minio-secret
              key: rootPassword
        ports:
        - containerPort: 9000
          name: api
        - containerPort: 9001
          name: console
        volumeMounts:
        - name: data
          mountPath: /data
        livenessProbe:
          httpGet:
            path: /minio/health/live
            port: 9000
          initialDelaySeconds: 30
          periodSeconds: 20
        readinessProbe:
          httpGet:
            path: /minio/health/ready
            port: 9000
          initialDelaySeconds: 15
          periodSeconds: 10
        resources:
          requests:
            memory: "512Mi"
            cpu: "250m"
          limits:
            memory: "2Gi"
            cpu: "1000m"
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: local-storage
      resources:
        requests:
          storage: 10Gi

as you can see, i am using hostPath storage with a local PersistentVolume for development environment.

Components:

1. Namespace: Dedicated minio namespace for isolation

2. Secret: Stores MinIO root credentials (change these in production!)

3. Headless Service: Required for StatefulSet pod identity

4. LoadBalancer Service: Exposes MinIO API (9000) and Console (9001)

5. StatefulSet: Main MinIO deployment with:

   • Single replica (scale for distributed mode)

   • Persistent volume claims

   • Health checks

   • Resource limits

Key Features:

• Persistent Storage: Uses volumeClaimTemplates to automatically create PVCs

• Stable Identity: Each pod gets a stable hostname and storage

• Health Probes: Liveness and readiness checks

Important Notes:

• Storage Class: Change storageClassName: standard to match your cluster's storage class

• Credentials: Update the Secret with strong passwords for production

• Scaling: For distributed MinIO, increase replicas to 4+ and adjust the server args

• Storage Size: Adjust the storage: 10Gi based on your needsfads

Deployment

kubectl apply -f minio-statefulset.yaml

Verify Deployment

Check pod & services:

# Check status
kubectl get pods -n minio
kubectl get pvc -n minio
kubectl get svc -n minio

# Access MinIO
# Get the LoadBalancer IP
kubectl get svc minio -n minio

jamilnoyda@Jamils-MacBook-Air BE % kubectl get pvc -n minio
NAME           STATUS   VOLUME       CAPACITY   ACCESS MODES   STORAGECLASS    VOLUMEATTRIBUTESCLASS   AGE
data-minio-0   Bound    minio-pv-0   10Gi       RWO            local-storage   <unset>                 12h
jamilnoyda@Jamils-MacBook-Air BE % 
jamilnoyda@Jamils-MacBook-Air BE % kubectl get svc -n minio
NAME             TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                         AGE
minio            LoadBalancer   10.100.126.178   localhost     9000:30817/TCP,9001:31619/TCP   12h
minio-headless   ClusterIP      None             <none>        9000/TCP,9001/TCP               12h
jamilnoyda@Jamils-MacBook-Air BE % kubectl get pods -n minio
NAME      READY   STATUS    RESTARTS   AGE
minio-0   1/1     Running   0          12h
jamilnoyda@Jamils-MacBook-Air BE % kubectl get statefulset -n minio
NAME    READY   AGE
minio   1/1     12h
jamilnoyda@Jamils-MacBook-Air BE %

Expected services:

• minio → API (port 9000)

• minio-console → Web UI (port 9001)

Access MinIO Console

Open in browser:

http://localhost:9001

Login with:

• Access Key: minioadmin

• Secret Key: minioadmin123

Voilà!

And there you go — you’ve built your very own S3 Bucket service that you can actually use!

Final Thoughts

Depending on your use case, you can make your own choice. Before reading this article, you probably didn’t have that option — but now you do.

There’s definitely a learning curve, when it comes to deployment, but. once it is deployed, their desgin and features are pretty much the same.

Signing Off- Jamil n.