diff --git a/content/docs/introduction/description.mdx b/content/docs/introduction/description.mdx index c33ea19a..ec03e392 100644 --- a/content/docs/introduction/description.mdx +++ b/content/docs/introduction/description.mdx @@ -7,22 +7,19 @@ The data centers operated by CESNET are based on object storage technology utili **CEPH** - Ceph is a software solution designed to provide object storage. Unlike traditional file-oriented data storage, object storage stores data as objects rather than data blocks. Metadata is directly associated with each object, eliminating the hierarchical structure used in conventional data storage systems. Instead, all data is stored in a flat namespace, making it highly scalable. Ceph is an open-source system that includes an object store, block store, and distributed file system. It supports object access (RADOS), exported block devices, and file access. Data stored in Ceph is replicated across multiple nodes based on a defined replication policy. -| Data Center | Type | Loacation | Raw capacity PB | In operation since | Status | -| :----- | :----- | :----- | :----- | :----- | :----- | -| DU1 | HSM | Plzeň | 3.6 | 2011 | decomissioned | -| DU2 | HSM | Jihlava | 7.209 | 2013 | decomissioned | -| DU3 | HSM | Brno | 3.52 | 2013 | decomissioned | -| DU4 | HSM | Ostrava | 23.341 | 2017 | offline | -| DU5 | RAID | Jihlava | 9.36 | 2019 | decomissioned | -| CL1 | CEPH | Jihlava | 10.56 | 2019 | replaced in 2025 | -| CL2 | CEPH | Plzeň | 25.433 | 2020 | online | -| CL3 | CEPH | Ostrava | 33.673 | 2021 | online | -| CL4 | CEPH | Brno | 26.607 | 2022 | online | -| CL5 | CEPH | Dolní Břežany | 25.536 | 2023 | online | -| CL1 | CEPH | Jihlava | 25.394 | 2025 | online | - ## Datacenters -Below are the details of each datacenter. + +Cesnet Data Storage Department currently operates the following datacenters: + +| Data Center | Type | Loacation | Raw capacity PB | In operation since | Status |S3 Endpoint Address| +| :----- | :----- | :----- | :----- | :----- | :----- |:----- | +| [CL1](#cl1-jihlava) | CEPH | Jihlava | 25.394 | 2025 | online | ```https://s3.cl1.du.cesnet.cz``` | +| [CL2](#cl2-plzeň) | CEPH | Plzeň | 25.433 | 2020 | online | ```https://s3.cl2.du.cesnet.cz``` | +| [CL3](#cl3-ostrava) | CEPH | Ostrava | 33.673 | 2021 | online | ```https://s3.cl3.du.cesnet.cz``` | +| [CL4](#cl4-brno) | CEPH | Brno | 26.607 | 2022 | online | ```https://s3.cl4.du.cesnet.cz``` | +| [CL5](#cl5-dolní-břežany) | CEPH | Dolní Břežany | 25.536 | 2023 | online | ```https://s3.cl5.du.cesnet.cz``` | + +--- ### CL1 Jihlava Object storage CL1 (modernised in 2025) is based on Ceph technology and consists of 34 storage servers, each equipped with 32 disks, totaling a capacity of 25.394 PB. Data access is provided by ten front-end servers. @@ -58,3 +55,19 @@ Object storage CL5 is based on Ceph technology and consists of 50 storage server Available services: [RBD](../object-storage-rbd/rbd-service) and [S3](../object-storage-s3/s3-service) --- + +## Decomissioned and Historical Datacenters + +| Data Center | Type | Loacation | Raw capacity PB | In operation since | Status | +| :----- | :----- | :----- | :----- | :----- | :----- | +| DU1 | HSM | Plzeň | 3.6 | 2011 | decomissioned | +| DU2 | HSM | Jihlava | 7.209 | 2013 | decomissioned | +| DU3 | HSM | Brno | 3.52 | 2013 | decomissioned | +| DU4 | HSM | Ostrava | 23.341 | 2017 | offline | +| DU5 | RAID | Jihlava | 9.36 | 2019 | decomissioned | +| CL1 | CEPH | Jihlava | 10.56 | 2019 | replaced in 2025 | + + + + + diff --git a/content/docs/introduction/introduction.mdx b/content/docs/introduction/introduction.mdx index c3517e17..112e07d6 100644 --- a/content/docs/introduction/introduction.mdx +++ b/content/docs/introduction/introduction.mdx @@ -30,7 +30,7 @@ For transferring files that are too large for email attachments to a single reci ### Data Sharing and Synchronisation -To keep your most important but smaller work data easily accessible and automatically synchronised across your computers and mobile devices, as well as available through a web interface, you can try out our [sync'n'share service](../collaboration/owncloud). +To keep your most important but smaller work data easily accessible and automatically synchronised across your computers and mobile devices, as well as available through a web interface, you can try out our [OwnCloud Drive service](../collaboration/owncloud). ### Object Storage for Individual Users diff --git a/content/docs/introduction/support.mdx b/content/docs/introduction/support.mdx index 6f0fd824..825d0725 100644 --- a/content/docs/introduction/support.mdx +++ b/content/docs/introduction/support.mdx @@ -12,7 +12,7 @@ Before submitting a request, please check the [FAQ](../faq/faq) to see if you ca The primary contact for most data storage-related issues is `support@cesnet.cz`. You can use this address for comments, suggestions, technical issues, or any technical questions intended for system administrators. -All correspondence received at the e-mail address mentioned above is recorded in the [Request Tracker](http://bestpractical.com/rt) system, which generates a ticket and assigns an ID number to it. +All correspondence received at the e-mail address mentioned above is recorded in the *Request Tracker* system, which generates a ticket and assigns an ID number to it. The author of the e-mail and all relevant data storage staff members responsible for resolving user issues are notified by e-mail when a ticket is created. The ticket ID will appear in the subject line of all e-mails related to the ticket. All e-mails are grouped under this ticket ID, so please retain this identification in the subject line when replying to data storage staff. diff --git a/content/docs/object-storage-s3/s3-clients.mdx b/content/docs/object-storage-s3/s3-clients.mdx index bdf9245c..1f5ccf0b 100644 --- a/content/docs/object-storage-s3/s3-clients.mdx +++ b/content/docs/object-storage-s3/s3-clients.mdx @@ -1,15 +1,8 @@ --- title: Favorite S3 service clients --- -The following section provides a list of recommended S3 clients. All S3 clients require the S3 credentials `access_key` and `secret_key` as well as the S3 endpoint address, see below. - - -**Available S3 endpoints**
- cl1 - https://s3.cl1.du.cesnet.cz
- cl2 - https://s3.cl2.du.cesnet.cz
- cl3 - https://s3.cl3.du.cesnet.cz
- cl4 - https://s3.cl4.du.cesnet.cz
- + +The following section provides a list of recommended S3 clients. All S3 clients require the S3 credentials `access_key` and `secret_key` as well as the [S3 endpoint address](../introduction/description.mdx#datacenters). ## S3 Browser (GUI Windows) [S3 Browser](https://s3browser.com/) is a free tool for Windows that allows you to manage your S3 storage, as well as upload and download data. It supports managing up to two user accounts (S3 account) at no cost. [The Guide for S3 Browser](./s3browser). diff --git a/content/docs/object-storage-s3/s3-getting-access.mdx b/content/docs/object-storage-s3/s3-getting-access.mdx index 8c1475a8..5372a643 100644 --- a/content/docs/object-storage-s3/s3-getting-access.mdx +++ b/content/docs/object-storage-s3/s3-getting-access.mdx @@ -2,15 +2,14 @@ title: Getting Access to CESNET S3 --- +--- ### Obtaining a User Account -The procedure to get access to the service depends on your needs. +The procedure to get access to the service depends on your situation. #### Access to an Existing Virtual Organisation -You may have been invited to an existing Virtual Organisation (VO) by the -VO manager. In that case, follow instructions given to you by the VO -manager to register. +You may have been invited to an existing Virtual Organisation (VO) by the VO manager. In that case, follow instructions given to you by the VO manager to register. #### Personal S3 Access @@ -18,43 +17,58 @@ Provided you are seeking an S3 service that - provides just a personal S3 space for your own use, - doesn't support sharing between users and/or groups, - has a standard quota of 2 TB (can be increased on request), - - but is available just upon simple registration (you need to be an - employee or a student of a Czech public university, the AS CR, or an employee of a CESNET customer; you are expected to declare your eligibility using the identity federation), -you are invited to register at https://einfra.cesnet.cz/allfed/registrar/?vo=VO_s3&group=s3_cl4. + - but is available just upon simple registration (you need to be an employee or a student of a Czech public university, the AS CR, or an employee of a CESNET customer; you are expected to declare your eligibility using the identity federation), +you are invited to [register for a personal S3 service in Perun](https://einfra.cesnet.cz/allfed/registrar/?vo=VO_s3&group=s3_cl4). -#### Group S3 Access, aka “Virtual Organisation” +#### Group S3 Access, aka *Virtual Organisation* -In case you need access to S3 and the conditions of Personal S3 Access above don't fit your needs, we will typically prepare a “Virtual Organisation” and appropriate S3 service for you individually. +In case you need access to S3 and the conditions of Personal S3 Access above don't fit your needs, we will typically prepare a *Virtual Organisation* and appropriate S3 service for you individually. Contact our support at `support@cesnet.cz` and we'll discuss your expectations with you. -You can also refer to [common S3 use cases](s3-use-cases) for inspiration about serving typical use cases. +Once the VO is created, you will become a VO manager - please refer to [user management system documentation](../perun/user-management) to learn how to create, invite and manage user accounts for your team. -### Accessing the Storage +*Note: You can also refer to [common S3 use cases](s3-use-cases) for inspiration about serving typical use cases.* + +--- + +### Creating Access Keys -When you finished your registration (or your brand new VO has been set up -for you), you will be notified by mail. Then you need to get credentials to -access the storage itself. +When you finished your registration (or your brand new VO has been set up for you and configured), you need to get credentials to access the storage itself. -#### User Account Management + + **Important:** It takes some time to propagate your Perun registration information to other systems. Please wait at least 30 minutes before you proceed with the following steps. + + +The storage access credentials consist of a pair of keys (long hexadecimal numbers) - an `access key` and a `secret key`; keys are generated by every user in the access control system called Gatekeeper [https://access.du.cesnet.cz](https://access.du.cesnet.cz). -You will need a suitable [S3 client](s3-clients), and to configure it with -your storage access credentials. +You can generate as many key pairs as you wish to access your storage. Is is recommended to generate and use a separate access/secret key pair for each and every specific tool you access the storage with (see below). - **Obtaining access credentials to the storage**
- Access credentials to most of our object storage facilities are available through https://access.du.cesnet.cz. + **Recommendation:** It might be a good practice to record key pairs together with a description of where they are to be used in a password management aplication (like Keepass, 1Password, etc.). While you can always find the `access key` at the Gatekeeper, there is no way to reveal the genereated `secret key` - it is displayed only once at the time of generation. In you lose the secret key, you have to generate a new pair of keys (and delete the old one once you have reconfigured all clients that used it). -Kindly note that personal S3 appears as “Personal Account” in the system; other Virtual Organisations are listed individually. We recommend using separate access/secret key pair for each and every specific tool you access the storage with. +To **generate keys**, log in to Gatekeeper [https://access.du.cesnet.cz](https://access.du.cesnet.cz), press the `+ New key` button and enter a name for the key pair (like *My WinSCP Keys*). Wait until keys are genereated and copy and save them **both**. Also, mark down (copy and save) the S3 Storage Endpoint address displayed above the list of generated keys. + + + **Note:** You will need access key, secret key, and S3 storage endpoint address to configure access to the storage in a client application. + + +Kindly note that personal S3 appears as *Personal Account* in the system; other Virtual Organisations are listed individually. + +--- + +### Accessing the Storage + +You will need to (install and) configure a suitable [S3 client](s3-clients) to acces your S3 storage. If in doubt, start with [WinSCP](winscp) or [S3 Browser](s3browser) on Windows, [Cyberduck](cyberduck) on Mac, or [rclone web interface](rclone#rclone-configuration-using-graphical-user-interface) on Linux. + +--- + +### Final Notes It is useful to distinguish following systems: - - Perun ([https://einfra.cesnet.cz/](https://einfra.cesnet.cz)) is a - system managing which users have access to particular services (based - on your membership in Virtual Organisations and groups) - - Gatekeeper ([https://access.du.cesnet.cz](https://access.du.cesnet.cz)] - will give you credentials to the storage based on what Perun says - - and finally, the storage itself is accessed with those credentials by a - client you prefer. + - Perun ([https://einfra.cesnet.cz/](https://einfra.cesnet.cz)) is a system managing which users have access to particular services (based on your membership in Virtual Organisations and groups) + - Gatekeeper ([https://access.du.cesnet.cz/](https://access.du.cesnet.cz)) will give you credentials to the storage based on what Perun says + - and finally, the storage itself is accessed with those credentials by a client you prefer.