Arpia Bucket Object Store S3 API Documentation

Table of Contents#

1. Key Features
2. Authentication
3. Endpoint Overview
4. Usage Summary
5. General Information
6. Detailed Endpoints * GET: List Files or Get a Specific File from Bucket * DELETE: Delete a File from the Bucket * PUT: Upload a File to the Bucket * POST: Work Area Project Submission
7. Using AutoAPI with Buckets

Key Features#

• File Management: List, retrieve, and delete files from a designated bucket.
• File Upload: Easily upload files directly to the bucket using base64 encoding or binary transfer, with instant access to the file link.
• Project Submission: Submit projects to a workspace, complete with debug mode for enhanced feedback.
• Flexible Upload Methods: Support for both base64-encoded and binary file uploads.

Authentication#

All requests require an authentication token. Include a valid token in the request query parameters to securely access the API.

Tokens can be created and managed using AutoAPI for seamless integration and workflow automation.

Endpoint Overview#

The Object Store S3 API supports the following HTTP methods:

Usage Summary#

Each endpoint allows for the following actions:

General Information#

File Size Limits#

• Maximum file size: 1 GB per file
• Total bucket storage: Varies by plan (check your ARPIA subscription)

Supported File Types#

• All file types are supported
• Common MIME types: image/*, application/json, application/pdf, text/*, video/*, audio/*

Rate Limiting#

• Standard requests: 100 requests per minute per token
• Large file uploads: 20 uploads per minute
• Exceeding limits returns HTTP 429 (Too Many Requests)

Important Notes#

• File names must be URL-safe (avoid special characters)
• Files with identical names will be overwritten
• All timestamps are in UTC format

Detailed Endpoints#

1. GET: List Files or Get a Specific File from Bucket#

• Endpoint:\ GET https://cloud.arpia.ai/api/bucket/

• Query Parameters:

• token (Required): Authentication token for accessing the API.
• bucket (Required): Unique identifier for the bucket, e.g., LElC1IWg.

• file (Optional): Name of the file to retrieve, e.g., upload_test.txt. If omitted, lists all files in the bucket.

• Description:\ Retrieve a list of all files in a bucket or get information about a specific file if the file parameter is specified.

• Example Request: GET https://cloud.arpia.ai/api/bucket/?token=YOUR_TOKEN&bucket=LElC1IWg

• Responses:

• 200 OK\ Request successful, with JSON response listing files or file data.

  json [ { "name": "upload_test.txt", "url": "https://store-pty1.datakubes.io/LElC1IWg/upload_test.txt", "size": 124652, "modified": "2023-12-13 14:05:18" }, { "name": "upload_test2.txt", "url": "https://store-pty1.datakubes.io/LElC1IWg/upload_test2.txt", "size": 124652, "modified": "2023-12-13 14:08:23" } ]

• 400 Bad Request\ Missing required parameters. json { "error": "YES", "error_message": "Missing required parameter: token or bucket" }

• 401 Unauthorized\ Invalid or expired authentication token. json { "error": "YES", "error_message": "Invalid authentication token" }

• 404 Not Found\ Bucket or file not found. json { "error": "YES", "error_message": "Bucket or file not found" }

• 429 Too Many Requests\ Rate limit exceeded. json { "error": "YES", "error_message": "Rate limit exceeded. Please try again later." }

2. DELETE: Delete a File from the Bucket#

• Endpoint:\ DELETE https://cloud.arpia.ai/api/bucket/

• Query Parameters:

• token (Required): Authentication token.
• bucket (Required): Unique identifier for the bucket, e.g., LElC1IWg.

• file (Required): Name of the file to delete, e.g., upload_test.txt.

• Description:\ Delete a specific file from the bucket. This action is permanent and cannot be undone.

• Example Request: DELETE https://cloud.arpia.ai/api/bucket/?token=YOUR_TOKEN&bucket=LElC1IWg&file=upload_test.txt

• Responses:

• 200 OK\ File deleted successfully. json { "result": "OK", "message": "File deleted successfully", "file": "upload_test.txt" }

• 400 Bad Request\ Missing required parameters. json { "error": "YES", "error_message": "Missing required parameter: file" }

• 401 Unauthorized\ Invalid authentication token. json { "error": "YES", "error_message": "Invalid authentication token" }

• 404 Not Found\ File not found in bucket. json { "error": "YES", "error_message": "File not found in bucket" }

• 405 Method Not Allowed\ Request method not supported for this resource. json { "error": "YES", "error_message": "This Request Method is not supported!" }

• 429 Too Many Requests\ Rate limit exceeded. json { "error": "YES", "error_message": "Rate limit exceeded" }

3. PUT: Upload a File to the Bucket#

• Endpoint:\ PUT https://cloud.arpia.ai/api/bucket/

• Query Parameters:

• token (Required): Authentication token.
• bucket (Required): Unique identifier for the bucket, e.g., LElC1IWg.
• file (Required): Name of the file being uploaded, e.g., upload_test.json.

• base64 (Optional): Set to y to indicate base64-encoded file content. Omit for binary upload.

• Description:\ Upload a file to the specified bucket. The API supports two upload methods:

1. Base64 Upload: Send file content as a base64-encoded string in JSON body
2. Binary Upload: Send raw file content directly in the request body

Method 1: Base64 Upload

When to use: For uploading files through web applications, JavaScript clients, or when the file content needs to be transmitted as text/JSON.

Endpoint Example:

PUT https://cloud.arpia.ai/api/bucket/?token=[YOUR_TOKEN]&bucket=[BUCKET_TOKEN]&base64=y&file=[FILE_NAME]

Request Headers:

• Content-Type: application/json

Request Body:

{
  "file": "iVBORw0KGgoAAAANSUhEUgAACK8AAAMFCAYAAAC4PkclAAAACXBIWXMAABcRAAAXEQHKJvM..."
}

JavaScript Example:

// Convert file to base64
const fileToBase64 = (file) => {
  return new Promise((resolve, reject) => {
    const reader = new FileReader();
    reader.readAsDataURL(file);
    reader.onload = () => resolve(reader.result.split(',')[1]);
    reader.onerror = error => reject(error);
  });
};

// Upload file
const uploadFile = async (file, token, bucket) => {
  const base64Content = await fileToBase64(file);

  const response = await fetch(
    `https://cloud.arpia.ai/api/bucket/?token=${token}&bucket=${bucket}&base64=y&file=${file.name}`,
    {
      method: 'PUT',
      headers: {
        'Content-Type': 'application/json'
      },
      body: JSON.stringify({ file: base64Content })
    }
  );

  return response.json();
};

Method 2: Binary Upload

When to use: For direct file uploads from command line, server-to-server transfers, or when maximum efficiency is needed.

Endpoint Example:

PUT https://cloud.arpia.ai/api/bucket/?token=[YOUR_TOKEN]&bucket=[BUCKET_TOKEN]&file=[FILE_NAME]

Request Headers:

• Content-Type: application/octet-stream (or the file's specific MIME type)

cURL Example:

curl --location --request PUT \
  'https://cloud.arpia.ai/api/bucket/?token=YOUR_TOKEN&bucket=BUCKET_TOKEN&file=example.json' \
  --header 'Content-Type: application/octet-stream' \
  --data-binary '@/path/to/your/file.json'

Python Example:

import requests

def upload_file(file_path, token, bucket, filename):
    url = f'https://cloud.arpia.ai/api/bucket/'
    params = {
        'token': token,
        'bucket': bucket,
        'file': filename
    }

    with open(file_path, 'rb') as f:
        response = requests.put(
            url,
            params=params,
            data=f,
            headers={'Content-Type': 'application/octet-stream'}
        )

    return response.json()

# Usage
result = upload_file('/path/to/file.json', 'YOUR_TOKEN', 'BUCKET_TOKEN', 'file.json')
print(result)

Response (Both Methods):

• 200 OK\ File uploaded successfully. json { "result": "OK", "file": "https://store-pty1.datakubes.io/LElC1IWg/upload_test.json", "message": "File uploaded successfully" }

• 400 Bad Request\ Missing required parameters or invalid file data. json { "error": "YES", "error_message": "Invalid file data or missing parameters" }

• 401 Unauthorized\ Invalid authentication token. json { "error": "YES", "error_message": "Invalid authentication token" }

• 413 Payload Too Large\ File exceeds maximum size limit (1 GB). json { "error": "YES", "error_message": "File size exceeds maximum limit of 1 GB" }

• 429 Too Many Requests\ Upload rate limit exceeded. json { "error": "YES", "error_message": "Upload rate limit exceeded" }

• 500 Internal Server Error\ Server-side error during upload. json { "error": "YES", "error_message": "Internal server error. Please try again later." }

4. POST: Work Area Project Submission#

• Endpoint:\ POST https://cloud.arpia.ai/api/

• Query Parameters:

• token (Required): Authentication token.
• wsp (Required): Workspace ID where the project will be submitted, e.g., hZBM6k4h.

• debug (Optional): Set to true to enable debug mode for detailed processing information.

• Request Body:

• json: Data payload for the project submission. json { "text": "Sample project data for submission.", "metadata": { "author": "John Doe", "version": "1.0" } }

• Description:\ Submit a project or workflow data to a specified workspace within the ARPIA platform. This endpoint is designed for integrating bucket operations with ARPIA's project management and workflow automation tools.

Use Cases:

• Submitting processed data from bucket files to a workspace
• Triggering workflows based on uploaded files

• Creating project records with associated bucket resources

• Example Request: POST https://cloud.arpia.ai/api/?token=YOUR_TOKEN&wsp=hZBM6k4h&debug=true

• Response:

• 200 OK\ Request successful, with JSON response detailing each step of the submission process. json [ { "Step": 1, "StepName": "First Step", "Msg": "Initial step for the action", "Error": "N", "Result": "DONE" }, { "Step": 2, "StepName": "Database", "Msg": "Database Transaction Query", "Error": "N", "Result": "DONE" }, { "Step": 3, "StepName": "File Process", "Msg": "File process success.", "Error": "N", "Result": "DONE" }, { "Step": 4, "StepName": "Relational Node Data Entry", "Msg": "Save to Relational Node Query", "Error": "N", "Result": "DONE" }, { "Step": 5, "FinalDBStep": "Y", "StepName": "Final DB Commit", "Msg": "Commit all database transactions", "Error": "N", "Result": "DONE" } ]

• 400 Bad Request\ Invalid request payload or missing parameters. json { "error": "YES", "error_message": "Invalid request body or missing workspace ID" }

• 401 Unauthorized\ Invalid authentication token. json { "error": "YES", "error_message": "Invalid authentication token" }

• 404 Not Found\ Workspace not found. json { "error": "YES", "error_message": "Workspace not found" }

Using AutoAPI with Buckets#

If you are using buckets within the ARPIA ecosystem, integrating them with the AutoAPI system can significantly enhance your workflow efficiency. The AutoAPI allows you to:

• Dynamically retrieve or update bucket data
• Automate file processing workflows
• Create custom endpoints that interact with your bucket resources
• Build event-driven architectures triggered by file uploads

Configuration Requirements#

When setting up AutoAPI endpoints for bucket operations:

1. Select the appropriate HTTP method (GET, POST, PUT, DELETE)
2. Link the AutoAPI to your object store in the configuration panel
3. Configure method-specific settings: * For GET: Define which files or metadata to retrieve * For PUT: Configure upload parameters and validation rules * For POST: Set up workspace integration and data transformation * For DELETE: Implement access controls and confirmation workflows

Common Integration Patterns#

Pattern 1: Automated File Processing

Upload a file to the bucket → AutoAPI triggers → Process file → Store results in workspace

Pattern 2: Data Synchronization

External system uploads to bucket → AutoAPI validates → Sync to database → Notify stakeholders

Pattern 3: Multi-Stage Workflows

Bucket upload → AutoAPI Stage 1 (validation) → AutoAPI Stage 2 (transformation) → Final storage

Important Configuration Notes#

⚠️ Common Misconfiguration: Selecting PUT method without properly linking the object store will cause setup failures. Always verify the bucket-to-AutoAPI linkage before deploying.

✅ Best Practice: Test your AutoAPI configuration with small files first, then enable debug mode to monitor the processing pipeline.

Additional Resources#

For comprehensive guidance on setting up and managing AutoAPI endpoints with buckets, including:

• Step-by-step configuration tutorials
• Advanced authentication options
• Rate limiting and throttling strategies
• Error handling best practices

See the AutoAPI Documentation.

Best Practices#

Security#

• Never expose tokens in client-side code - Use server-side proxies for web applications
• Rotate tokens regularly - Update authentication tokens periodically
• Implement access controls - Use workspace-level permissions to restrict bucket access
• Validate file types - Check file extensions and MIME types before upload

Performance#

• Use binary uploads for large files - More efficient than base64 encoding
• Implement retry logic - Handle rate limiting and temporary failures gracefully
• Batch operations - Group multiple operations when possible
• Monitor rate limits - Track API usage to avoid throttling

Error Handling#

• Always check response status codes - Don't assume success
• Implement exponential backoff - For retry strategies after rate limiting
• Log detailed error messages - Include request IDs when contacting support
• Validate before upload - Check file size and format client-side to reduce errors

File Management#

• Use descriptive file names - Include timestamps or unique identifiers
• Implement versioning - Track file versions in your application logic
• Clean up old files - Implement retention policies to manage storage costs
• Document file structures - Maintain metadata about bucket contents

Troubleshooting#

Common Issues#

Problem: "Invalid authentication token" error\ Solution: Verify your token is current and hasn't expired. Generate a new token from the AutoAPI object-storage configuration.

Problem: Files not appearing after upload\ Solution: Check the upload response for success confirmation. Verify bucket ID is correct. Allow a few seconds for indexing.

Problem: Base64 upload fails with large files\ Solution: Use binary upload method for files > 50 MB. Base64 encoding increases payload size by \~33%, and large base64 strings can cause timeout or memory issues.

Problem: Rate limit errors during bulk uploads\ Solution: Implement request queuing with delays between uploads. Consider increasing rate limits for your plan.

Support#

For additional assistance:

• Documentation: https://docs.arpia.ai
• Community Forum: https://community.arpia.ai
• Technical Support: support@arpia.ai
• Status Page: https://status.arpia.ai

Last Updated: December 2024\ API Version: 2.0\ Document Version: 2.1