kudzues/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-04-05 10:25:48 +08:00
Code Issues Packages Projects Releases Wiki Activity

refactor(i18n): use JSON with flattened key and namespace (#30114)

Browse Source 
Co-authored-by: yyh <yuanyouhuilyz@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
This commit is contained in:
Stephen Zhou
2025-12-29 14:52:32 +08:00
committed by GitHub
parent 09be869f58
commit 6d0e36479b
2552 changed files with 111159 additions and 142972 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
web/i18n-config/DEV.md
View File

@@ -23,7 +23,7 @@
- `setLocaleOnClient`
- `changeLanguage` (defined in i18n/i18next-config, also init i18n resources (side effects))
- is `i18next.changeLanguage`
- all languages text is merge & load in FrontEnd as .js (see i18n/i18next-config)
- loads JSON namespaces for the target locale and merges resource bundles (see i18n/i18next-config)
- i18n context
- `locale` - current locale code (ex `eu-US`, `zh-Hans`)
- `i18n` - useless
@@ -32,13 +32,16 @@
### load i18n resources
- client: i18n/i18next-config.ts
- ns = camalCase(filename)
- ns = camelCase(filename) (app-debug -> appDebug)
- keys are flat (dot notation); `keySeparator: false`
- ex: `app/components/datasets/create/embedding-process/index.tsx`
- `t('datasetSettings.form.retrievalSetting.title')`
- `const { t } = useTranslation('datasetSettings')`
- `t('form.retrievalSetting.title')`
- server: i18n/server.ts
- ns = filename
- ns = filename (kebab-case) mapped to camelCase namespace
- ex: `app/(commonLayout)/datasets/(datasetDetailLayout)/[datasetId]/settings/page.tsx`
- `translate(locale, 'dataset-settings')`
- `const { t } = await getTranslation(locale, 'dataset-settings')`
- `t('form.retrievalSetting.title')`
## TODO

71
web/i18n-config/README.md
View File

@@ -2,43 +2,34 @@
## Introduction
This directory contains the internationalization (i18n) files for this project.
This directory contains i18n tooling and configuration. Translation files live under `web/i18n`.
## File Structure
```
├── [ 24] README.md
├── [ 704] en-US
│   ├── [2.4K] app-annotation.ts
│   ├── [5.2K] app-api.ts
│   ├── [ 16K] app-debug.ts
│   ├── [2.1K] app-log.ts
│   ├── [5.3K] app-overview.ts
│   ├── [1.9K] app.ts
│   ├── [4.1K] billing.ts
│   ├── [ 17K] common.ts
│   ├── [ 859] custom.ts
│   ├── [5.7K] dataset-creation.ts
│   ├── [ 10K] dataset-documents.ts
│   ├── [ 761] dataset-hit-testing.ts
│   ├── [1.7K] dataset-settings.ts
│   ├── [2.0K] dataset.ts
│   ├── [ 941] explore.ts
│   ├── [ 52] layout.ts
│   ├── [2.3K] login.ts
│   ├── [ 52] register.ts
│   ├── [2.5K] share.ts
│   └── [2.8K] tools.ts
├── [1.6K] i18next-config.ts
├── [ 634] index.ts
├── [4.4K] language.ts
web/i18n
├── en-US
│ ├── app.json
│ ├── app-debug.json
│ ├── common.json
│ └── ...
└── zh-Hans
└── ...
web/i18n-config
├── auto-gen-i18n.js
├── check-i18n.js
├── i18next-config.ts
└── ...
```
We use English as the default language. The i18n files are organized by language and then by module. For example, the English translation for the `app` module is in `en-US/app.ts`.
We use English as the default language. Translation files are organized by language and then by module. For example, the English translation for the `app` module is in `web/i18n/en-US/app.json`.
If you want to add a new language or modify an existing translation, you can create a new file for the language or modify the existing file. The file name should be the language code (e.g., `zh-Hans` for Chinese) and the file extension should be `.ts`.
Translation files are JSON with flat keys (dot notation). i18next is configured with `keySeparator: false`, so dots are part of the key. The namespace is the camelCase file name (for example, `app-debug.json` -> `appDebug`), so use `useTranslation('appDebug')` or `t('key', { ns: 'appDebug' })`.
For example, if you want to add french translation, you can create a new folder `fr-FR` and add the translation files in it.
If you want to add a new language or modify an existing translation, create or update the `.json` files in the language folder.
For example, if you want to add French translation, you can create a new folder `fr-FR` and add the translation files in it.
By default we will use `LanguagesSupported` to determine which languages are supported. For example, in login page and settings page, we will use `LanguagesSupported` to determine which languages are supported and display them in the language selection dropdown.
@@ -51,13 +42,9 @@ cd web/i18n
cp -r en-US id-ID
```
2. Modify the translation files in the new folder.
2. Modify the translation `.json` files in the new folder. Keep keys flat (for example, `dialog.title`).
1. Add type to new language in the `language.ts` file.
> Note: `I18nText` type is now automatically derived from `LanguagesSupported`, so you don't need to manually add types.
4. Add the new language to the `languages.ts` file.
1. Add the new language to the `languages.ts` file.
```typescript
export const languages = [
@@ -157,16 +144,18 @@ export const languages = [
]
```
5. Don't forget to mark the supported field as `true` if the language is supported.
4. Don't forget to mark the supported field as `true` if the language is supported.
1. Sometime you might need to do some changes in the server side. Please change this file as well. 👇
1. Sometimes you might need to do some changes in the server side. Please change this file as well. 👇
https://github.com/langgenius/dify/blob/61e4bbabaf2758354db4073cbea09fdd21a5bec1/api/constants/languages.py#L5
> Note: `I18nText` type is automatically derived from `LanguagesSupported`, so you don't need to manually add types.
## Clean Up
That's it! You have successfully added a new language to the project. If you want to remove a language, you can simply delete the folder and remove the language from the `language.ts` file.
That's it! You have successfully added a new language to the project. If you want to remove a language, you can simply delete the folder and remove the language from the `languages.ts` file.
We have a list of languages that we support in the `language.ts` file. But some of them are not supported yet. So, they are marked as `false`. If you want to support a language, you can follow the steps above and mark the supported field as `true`.
We have a list of languages that we support in the `languages.ts` file. But some of them are not supported yet. So, they are marked as `false`. If you want to support a language, you can follow the steps above and mark the supported field as `true`.
## Utility scripts
@@ -174,6 +163,6 @@ We have a list of languages that we support in the `language.ts` file. But some
- Use space-separated values; repeat `--file` / `--lang` as needed. Defaults to all en-US files and all supported locales except en-US.
- Protects placeholders (`{{var}}`, `${var}`, `<tag>`) before translation and restores them after.
- Check missing/extra keys: `pnpm run check-i18n --file app billing --lang zh-Hans [--auto-remove]`
- Use space-separated values; repeat `--file` / `--lang` as needed. Returns non-zero on missing/extra keys (CI will fail); `--auto-remove` deletes extra keys automatically.
- Use space-separated values; repeat `--file` / `--lang` as needed. Returns non-zero on missing/extra keys; `--auto-remove` deletes extra keys automatically.
Workflows: `.github/workflows/translate-i18n-base-on-english.yml` auto-runs the translation generator on en-US changes to main; `.github/workflows/web-tests.yml` checks i18n keys on web changes.
Workflows: `.github/workflows/translate-i18n-base-on-english.yml` auto-runs the translation generator on `web/i18n/en-US/*.json` changes to main. `check-i18n` is a manual script (not run in CI).

145
web/i18n-config/auto-gen-i18n.js
View File

@@ -1,14 +1,9 @@
import fs from 'node:fs'
import { createRequire } from 'node:module'
import path from 'node:path'
import { fileURLToPath } from 'node:url'
import vm from 'node:vm'
import { translate } from 'bing-translate-api'
import { generateCode, loadFile, parseModule } from 'magicast'
import { transpile } from 'typescript'
import data from './languages'
const require = createRequire(import.meta.url)
const __filename = fileURLToPath(import.meta.url)
const __dirname = path.dirname(__filename)
@@ -167,136 +162,48 @@ async function translateText(source, toLanguage) {
}
}
async function translateMissingKeyDeeply(sourceObj, targetObject, toLanguage) {
async function translateMissingKeys(sourceObj, targetObject, toLanguage) {
const skippedKeys = []
const translatedKeys = []
const entries = Object.keys(sourceObj)
const processArray = async (sourceArray, targetArray, parentKey) => {
for (let i = 0; i < sourceArray.length; i++) {
const item = sourceArray[i]
const pathKey = `${parentKey}[${i}]`
const existingTarget = targetArray[i]
if (typeof item === 'object' && item !== null) {
const targetChild = (Array.isArray(existingTarget) || typeof existingTarget === 'object') ? existingTarget : (Array.isArray(item) ? [] : {})
const childResult = await translateMissingKeyDeeply(item, targetChild, toLanguage)
targetArray[i] = targetChild
skippedKeys.push(...childResult.skipped.map(k => `${pathKey}.${k}`))
translatedKeys.push(...childResult.translated.map(k => `${pathKey}.${k}`))
}
else {
if (existingTarget !== undefined)
continue
const translationResult = await translateText(item, toLanguage)
targetArray[i] = translationResult.value ?? ''
if (translationResult.skipped)
skippedKeys.push(`${pathKey}: ${item}`)
else
translatedKeys.push(pathKey)
}
}
}
for (const key of entries) {
for (const key of Object.keys(sourceObj)) {
const sourceValue = sourceObj[key]
const targetValue = targetObject[key]
if (targetValue === undefined) {
if (Array.isArray(sourceValue)) {
const translatedArray = []
await processArray(sourceValue, translatedArray, key)
targetObject[key] = translatedArray
}
else if (typeof sourceValue === 'object' && sourceValue !== null) {
targetObject[key] = {}
const result = await translateMissingKeyDeeply(sourceValue, targetObject[key], toLanguage)
skippedKeys.push(...result.skipped.map(k => `${key}.${k}`))
translatedKeys.push(...result.translated.map(k => `${key}.${k}`))
}
else {
const translationResult = await translateText(sourceValue, toLanguage)
targetObject[key] = translationResult.value ?? ''
if (translationResult.skipped)
skippedKeys.push(`${key}: ${sourceValue}`)
else
translatedKeys.push(key)
}
}
else if (Array.isArray(sourceValue)) {
const targetArray = Array.isArray(targetValue) ? targetValue : []
await processArray(sourceValue, targetArray, key)
targetObject[key] = targetArray
}
else if (typeof sourceValue === 'object' && sourceValue !== null) {
const targetChild = targetValue && typeof targetValue === 'object' ? targetValue : {}
targetObject[key] = targetChild
const result = await translateMissingKeyDeeply(sourceValue, targetChild, toLanguage)
skippedKeys.push(...result.skipped.map(k => `${key}.${k}`))
translatedKeys.push(...result.translated.map(k => `${key}.${k}`))
}
else {
// Overwrite when type is different or value is missing to keep structure in sync
const shouldUpdate = typeof targetValue !== typeof sourceValue || targetValue === undefined || targetValue === null
if (shouldUpdate) {
const translationResult = await translateText(sourceValue, toLanguage)
targetObject[key] = translationResult.value ?? ''
if (translationResult.skipped)
skippedKeys.push(`${key}: ${sourceValue}`)
else
translatedKeys.push(key)
}
}
// Skip if target already has this key
if (targetValue !== undefined)
continue
const translationResult = await translateText(sourceValue, toLanguage)
targetObject[key] = translationResult.value ?? ''
if (translationResult.skipped)
skippedKeys.push(`${key}: ${sourceValue}`)
else
translatedKeys.push(key)
}
return { skipped: skippedKeys, translated: translatedKeys }
}
async function autoGenTrans(fileName, toGenLanguage, isDryRun = false) {
const fullKeyFilePath = path.resolve(__dirname, i18nFolder, targetLanguage, `${fileName}.ts`)
const toGenLanguageFilePath = path.resolve(__dirname, i18nFolder, toGenLanguage, `${fileName}.ts`)
const fullKeyFilePath = path.resolve(__dirname, i18nFolder, targetLanguage, `${fileName}.json`)
const toGenLanguageFilePath = path.resolve(__dirname, i18nFolder, toGenLanguage, `${fileName}.json`)
try {
const content = fs.readFileSync(fullKeyFilePath, 'utf8')
// Create a safer module environment for vm
const moduleExports = {}
const context = {
exports: moduleExports,
module: { exports: moduleExports },
require,
console,
__filename: fullKeyFilePath,
__dirname: path.dirname(fullKeyFilePath),
}
// Use vm.runInNewContext instead of eval for better security
vm.runInNewContext(transpile(content), context)
const fullKeyContent = moduleExports.default || moduleExports
const fullKeyContent = JSON.parse(content)
if (!fullKeyContent || typeof fullKeyContent !== 'object')
throw new Error(`Failed to extract translation object from ${fullKeyFilePath}`)
// if toGenLanguageFilePath is not exist, create it
if (!fs.existsSync(toGenLanguageFilePath)) {
fs.writeFileSync(toGenLanguageFilePath, `const translation = {
}
export default translation
`)
// if toGenLanguageFilePath does not exist, create it with empty object
let toGenOutPut = {}
if (fs.existsSync(toGenLanguageFilePath)) {
const existingContent = fs.readFileSync(toGenLanguageFilePath, 'utf8')
toGenOutPut = JSON.parse(existingContent)
}
// To keep object format and format it for magicast to work: const translation = { ... } => export default {...}
const readContent = await loadFile(toGenLanguageFilePath)
const { code: toGenContent } = generateCode(readContent)
const mod = await parseModule(`export default ${toGenContent.replace('export default translation', '').replace('const translation = ', '')}`)
const toGenOutPut = mod.exports.default
console.log(`\n🌍 Processing ${fileName} for ${toGenLanguage}...`)
const result = await translateMissingKeyDeeply(fullKeyContent, toGenOutPut, toGenLanguage)
const result = await translateMissingKeys(fullKeyContent, toGenOutPut, toGenLanguage)
// Generate summary report
console.log(`\n📊 Translation Summary for ${fileName} -> ${toGenLanguage}:`)
@@ -310,11 +217,7 @@ export default translation
console.log(` ... and ${result.skipped.length - 5} more`)
}
const { code } = generateCode(mod)
const res = `const translation =${code.replace('export default', '')}
export default translation
`.replace(/,\n\n/g, ',\n').replace('};', '}')
const res = `${JSON.stringify(toGenOutPut, null, 2)}\n`
if (!isDryRun) {
fs.writeFileSync(toGenLanguageFilePath, res)
@@ -361,8 +264,8 @@ async function main() {
const filesInEn = fs
.readdirSync(path.resolve(__dirname, i18nFolder, targetLanguage))
.filter(file => /\.ts$/.test(file)) // Only process .ts files
.map(file => file.replace(/\.ts$/, ''))
.filter(file => /\.json$/.test(file)) // Only process .json files
.map(file => file.replace(/\.json$/, ''))
// Filter by target files if specified
const filesToProcess = targetFiles.length > 0 ? filesInEn.filter(f => targetFiles.includes(f)) : filesInEn

185
web/i18n-config/check-i18n.js
View File

@@ -1,12 +1,8 @@
import fs from 'node:fs'
import { createRequire } from 'node:module'
import path from 'node:path'
import { fileURLToPath } from 'node:url'
import vm from 'node:vm'
import { transpile } from 'typescript'
import data from './languages'
const require = createRequire(import.meta.url)
const __filename = fileURLToPath(import.meta.url)
const __dirname = path.dirname(__filename)
@@ -119,34 +115,18 @@ async function getKeysFromLanguage(language) {
return
}
// Filter only .ts and .js files
const translationFiles = files.filter(file => /\.(ts|js)$/.test(file))
// Filter only .json files
const translationFiles = files.filter(file => /\.json$/.test(file))
translationFiles.forEach((file) => {
const filePath = path.join(folderPath, file)
const fileName = file.replace(/\.[^/.]+$/, '') // Remove file extension
const fileName = file.replace(/\.json$/, '') // Remove file extension
const camelCaseFileName = fileName.replace(/[-_](.)/g, (_, c) =>
c.toUpperCase()) // Convert to camel case
try {
const content = fs.readFileSync(filePath, 'utf8')
// Create a safer module environment for vm
const moduleExports = {}
const context = {
exports: moduleExports,
module: { exports: moduleExports },
require,
console,
__filename: filePath,
__dirname: folderPath,
}
// Use vm.runInNewContext instead of eval for better security
vm.runInNewContext(transpile(content), context)
// Extract the translation object
const translationObj = moduleExports.default || moduleExports
const translationObj = JSON.parse(content)
if (!translationObj || typeof translationObj !== 'object') {
console.error(`Error parsing file: ${filePath}`)
@@ -154,24 +134,8 @@ async function getKeysFromLanguage(language) {
return
}
const nestedKeys = []
const iterateKeys = (obj, prefix = '') => {
for (const key in obj) {
const nestedKey = prefix ? `${prefix}.${key}` : key
if (typeof obj[key] === 'object' && obj[key] !== null && !Array.isArray(obj[key])) {
// This is an object (but not array), recurse into it but don't add it as a key
iterateKeys(obj[key], nestedKey)
}
else {
// This is a leaf node (string, number, boolean, array, etc.), add it as a key
nestedKeys.push(nestedKey)
}
}
}
iterateKeys(translationObj)
// Fixed: accumulate keys instead of overwriting
const fileKeys = nestedKeys.map(key => `${camelCaseFileName}.${key}`)
// Flat structure: just get all keys directly
const fileKeys = Object.keys(translationObj).map(key => `${camelCaseFileName}.${key}`)
allKeys.push(...fileKeys)
}
catch (error) {
@@ -185,7 +149,7 @@ async function getKeysFromLanguage(language) {
}
async function removeExtraKeysFromFile(language, fileName, extraKeys) {
const filePath = path.resolve(__dirname, '../i18n', language, `${fileName}.ts`)
const filePath = path.resolve(__dirname, '../i18n', language, `${fileName}.json`)
if (!fs.existsSync(filePath)) {
console.log(`⚠️ File not found: ${filePath}`)
@@ -204,116 +168,17 @@ async function removeExtraKeysFromFile(language, fileName, extraKeys) {
console.log(`🔄 Processing file: ${filePath}`)
// Read the original file content
// Read and parse JSON
const content = fs.readFileSync(filePath, 'utf8')
const lines = content.split('\n')
const translationObj = JSON.parse(content)
let modified = false
const linesToRemove = []
// Find lines to remove for each key (including multiline values)
// Remove each extra key (flat structure - direct property deletion)
for (const keyToRemove of fileSpecificKeys) {
const keyParts = keyToRemove.split('.')
let targetLineIndex = -1
const linesToRemoveForKey = []
// Build regex pattern for the exact key path
if (keyParts.length === 1) {
// Simple key at root level like "pickDate: 'value'"
for (let i = 0; i < lines.length; i++) {
const line = lines[i]
const simpleKeyPattern = new RegExp(`^\\s*${keyParts[0]}\\s*:`)
if (simpleKeyPattern.test(line)) {
targetLineIndex = i
break
}
}
}
else {
// Nested key - need to find the exact path
const currentPath = []
let braceDepth = 0
for (let i = 0; i < lines.length; i++) {
const line = lines[i]
const trimmedLine = line.trim()
// Track current object path
const keyMatch = trimmedLine.match(/^(\w+)\s*:\s*\{/)
if (keyMatch) {
currentPath.push(keyMatch[1])
braceDepth++
}
else if (trimmedLine === '},' || trimmedLine === '}') {
if (braceDepth > 0) {
braceDepth--
currentPath.pop()
}
}
// Check if this line matches our target key
const leafKeyMatch = trimmedLine.match(/^(\w+)\s*:/)
if (leafKeyMatch) {
const fullPath = [...currentPath, leafKeyMatch[1]]
const fullPathString = fullPath.join('.')
if (fullPathString === keyToRemove) {
targetLineIndex = i
break
}
}
}
}
if (targetLineIndex !== -1) {
linesToRemoveForKey.push(targetLineIndex)
// Check if this is a multiline key-value pair
const keyLine = lines[targetLineIndex]
const trimmedKeyLine = keyLine.trim()
// If key line ends with ":" (not ":", "{ " or complete value), it's likely multiline
if (trimmedKeyLine.endsWith(':') && !trimmedKeyLine.includes('{') && !trimmedKeyLine.match(/:\s*['"`]/)) {
// Find the value lines that belong to this key
let currentLine = targetLineIndex + 1
let foundValue = false
while (currentLine < lines.length) {
const line = lines[currentLine]
const trimmed = line.trim()
// Skip empty lines
if (trimmed === '') {
currentLine++
continue
}
// Check if this line starts a new key (indicates end of current value)
if (trimmed.match(/^\w+\s*:/))
break
// Check if this line is part of the value
if (trimmed.startsWith('\'') || trimmed.startsWith('"') || trimmed.startsWith('`') || foundValue) {
linesToRemoveForKey.push(currentLine)
foundValue = true
// Check if this line ends the value (ends with quote and comma/no comma)
if ((trimmed.endsWith('\',') || trimmed.endsWith('",') || trimmed.endsWith('`,')
|| trimmed.endsWith('\'') || trimmed.endsWith('"') || trimmed.endsWith('`'))
&& !trimmed.startsWith('//')) {
break
}
}
else {
break
}
currentLine++
}
}
linesToRemove.push(...linesToRemoveForKey)
console.log(`🗑️ Found key to remove: ${keyToRemove} at line ${targetLineIndex + 1}${linesToRemoveForKey.length > 1 ? ` (multiline, ${linesToRemoveForKey.length} lines)` : ''}`)
if (keyToRemove in translationObj) {
delete translationObj[keyToRemove]
console.log(`🗑️ Removed key: ${keyToRemove}`)
modified = true
}
else {
@@ -322,26 +187,8 @@ async function removeExtraKeysFromFile(language, fileName, extraKeys) {
}
if (modified) {
// Remove duplicates and sort in reverse order to maintain correct indices
const uniqueLinesToRemove = [...new Set(linesToRemove)].sort((a, b) => b - a)
for (const lineIndex of uniqueLinesToRemove) {
const line = lines[lineIndex]
console.log(`🗑️ Removing line ${lineIndex + 1}: ${line.trim()}`)
lines.splice(lineIndex, 1)
// Also remove trailing comma from previous line if it exists and the next line is a closing brace
if (lineIndex > 0 && lineIndex < lines.length) {
const prevLine = lines[lineIndex - 1]
const nextLine = lines[lineIndex] ? lines[lineIndex].trim() : ''
if (prevLine.trim().endsWith(',') && (nextLine.startsWith('}') || nextLine === ''))
lines[lineIndex - 1] = prevLine.replace(/,\s*$/, '')
}
}
// Write back to file
const newContent = lines.join('\n')
const newContent = `${JSON.stringify(translationObj, null, 2)}\n`
fs.writeFileSync(filePath, newContent)
console.log(`💾 Updated file: ${filePath}`)
return true
@@ -416,8 +263,8 @@ async function main() {
// Get all translation files
const i18nFolder = path.resolve(__dirname, '../i18n', language)
const files = fs.readdirSync(i18nFolder)
.filter(file => /\.ts$/.test(file))
.map(file => file.replace(/\.ts$/, ''))
.filter(file => /\.json$/.test(file))
.map(file => file.replace(/\.json$/, ''))
.filter(f => targetFiles.length === 0 || targetFiles.includes(f))
let totalRemoved = 0

115
web/i18n-config/i18next-config.ts
View File

@@ -4,39 +4,39 @@ import { camelCase, kebabCase } from 'es-toolkit/compat'
import i18n from 'i18next'
import { initReactI18next } from 'react-i18next'
import app from '../i18n/en-US/app'
import appAnnotation from '../i18n/en-US/app-annotation'
import appApi from '../i18n/en-US/app-api'
import appDebug from '../i18n/en-US/app-debug'
import appLog from '../i18n/en-US/app-log'
import appOverview from '../i18n/en-US/app-overview'
import billing from '../i18n/en-US/billing'
import common from '../i18n/en-US/common'
import custom from '../i18n/en-US/custom'
import dataset from '../i18n/en-US/dataset'
import datasetCreation from '../i18n/en-US/dataset-creation'
import datasetDocuments from '../i18n/en-US/dataset-documents'
import datasetHitTesting from '../i18n/en-US/dataset-hit-testing'
import datasetPipeline from '../i18n/en-US/dataset-pipeline'
import datasetSettings from '../i18n/en-US/dataset-settings'
import education from '../i18n/en-US/education'
import explore from '../i18n/en-US/explore'
import layout from '../i18n/en-US/layout'
import login from '../i18n/en-US/login'
import oauth from '../i18n/en-US/oauth'
import pipeline from '../i18n/en-US/pipeline'
import plugin from '../i18n/en-US/plugin'
import pluginTags from '../i18n/en-US/plugin-tags'
import pluginTrigger from '../i18n/en-US/plugin-trigger'
import register from '../i18n/en-US/register'
import runLog from '../i18n/en-US/run-log'
import share from '../i18n/en-US/share'
import time from '../i18n/en-US/time'
import tools from '../i18n/en-US/tools'
import workflow from '../i18n/en-US/workflow'
import appAnnotation from '../i18n/en-US/app-annotation.json'
import appApi from '../i18n/en-US/app-api.json'
import appDebug from '../i18n/en-US/app-debug.json'
import appLog from '../i18n/en-US/app-log.json'
import appOverview from '../i18n/en-US/app-overview.json'
import app from '../i18n/en-US/app.json'
import billing from '../i18n/en-US/billing.json'
import common from '../i18n/en-US/common.json'
import custom from '../i18n/en-US/custom.json'
import datasetCreation from '../i18n/en-US/dataset-creation.json'
import datasetDocuments from '../i18n/en-US/dataset-documents.json'
import datasetHitTesting from '../i18n/en-US/dataset-hit-testing.json'
import datasetPipeline from '../i18n/en-US/dataset-pipeline.json'
import datasetSettings from '../i18n/en-US/dataset-settings.json'
import dataset from '../i18n/en-US/dataset.json'
import education from '../i18n/en-US/education.json'
import explore from '../i18n/en-US/explore.json'
import layout from '../i18n/en-US/layout.json'
import login from '../i18n/en-US/login.json'
import oauth from '../i18n/en-US/oauth.json'
import pipeline from '../i18n/en-US/pipeline.json'
import pluginTags from '../i18n/en-US/plugin-tags.json'
import pluginTrigger from '../i18n/en-US/plugin-trigger.json'
import plugin from '../i18n/en-US/plugin.json'
import register from '../i18n/en-US/register.json'
import runLog from '../i18n/en-US/run-log.json'
import share from '../i18n/en-US/share.json'
import time from '../i18n/en-US/time.json'
import tools from '../i18n/en-US/tools.json'
import workflow from '../i18n/en-US/workflow.json'
// @keep-sorted
export const messagesEN = {
export const resources = {
app,
appAnnotation,
appApi,
@@ -69,8 +69,6 @@ export const messagesEN = {
workflow,
}
// pluginTrigger -> plugin-trigger
export type KebabCase<S extends string> = S extends `${infer T}${infer U}`
? T extends Lowercase<T>
? `${T}${KebabCase<U>}`
@@ -81,40 +79,45 @@ export type CamelCase<S extends string> = S extends `${infer T}-${infer U}`
? `${T}${Capitalize<CamelCase<U>>}`
: S
export type KeyPrefix = keyof typeof messagesEN
export type Namespace = KebabCase<KeyPrefix>
export type Resources = typeof resources
export type NamespaceCamelCase = keyof Resources
export type NamespaceKebabCase = KebabCase<NamespaceCamelCase>
const requireSilent = async (lang: Locale, namespace: Namespace) => {
const requireSilent = async (lang: Locale, namespace: NamespaceKebabCase) => {
let res
try {
res = (await import(`../i18n/${lang}/${namespace}`)).default
res = (await import(`../i18n/${lang}/${namespace}.json`)).default
}
catch {
res = (await import(`../i18n/en-US/${namespace}`)).default
res = (await import(`../i18n/en-US/${namespace}.json`)).default
}
return res
}
const NAMESPACES = Object.keys(messagesEN).map(kebabCase) as Namespace[]
const NAMESPACES = Object.keys(resources).map(kebabCase) as NamespaceKebabCase[]
export const loadLangResources = async (lang: Locale) => {
const modules = await Promise.all(
NAMESPACES.map(ns => requireSilent(lang, ns)),
)
const resources = modules.reduce((acc, mod, index) => {
acc[camelCase(NAMESPACES[index])] = mod
return acc
}, {} as Record<string, any>)
return resources
// Load a single namespace for a language
export const loadNamespace = async (lang: Locale, ns: NamespaceKebabCase) => {
const camelNs = camelCase(ns) as NamespaceCamelCase
if (i18n.hasResourceBundle(lang, camelNs))
return
const resource = await requireSilent(lang, ns)
i18n.addResourceBundle(lang, camelNs, resource, true, true)
}
// Load en-US resources first to make sure fallback works
// Load all namespaces for a language (used when switching language)
export const loadLangResources = async (lang: Locale) => {
await Promise.all(
NAMESPACES.map(ns => loadNamespace(lang, ns)),
)
}
// Initial resources: load en-US namespaces for fallback/default locale
const getInitialTranslations = () => {
return {
'en-US': {
translation: messagesEN,
},
'en-US': resources,
}
}
@@ -123,16 +126,16 @@ if (!i18n.isInitialized) {
lng: undefined,
fallbackLng: 'en-US',
resources: getInitialTranslations(),
defaultNS: 'common',
ns: Object.keys(resources),
keySeparator: false,
})
}
export const changeLanguage = async (lng?: Locale) => {
if (!lng)
return
if (!i18n.hasResourceBundle(lng, 'translation')) {
const resource = await loadLangResources(lng)
i18n.addResourceBundle(lng, 'translation', resource, true, true)
}
await loadLangResources(lng)
await i18n.changeLanguage(lng)
}

17
web/i18n-config/server.ts
View File

@@ -1,6 +1,7 @@
import type { Locale } from '.'
import type { Namespace } from './i18next-config'
import type { NamespaceCamelCase, NamespaceKebabCase } from './i18next-config'
import { match } from '@formatjs/intl-localematcher'
import { camelCase } from 'es-toolkit/compat'
import { createInstance } from 'i18next'
import resourcesToBackend from 'i18next-resources-to-backend'
import Negotiator from 'negotiator'
@@ -9,24 +10,28 @@ import { initReactI18next } from 'react-i18next/initReactI18next'
import { i18n } from '.'
// https://locize.com/blog/next-13-app-dir-i18n/
const initI18next = async (lng: Locale, ns: Namespace) => {
const initI18next = async (lng: Locale, ns: NamespaceKebabCase) => {
const i18nInstance = createInstance()
await i18nInstance
.use(initReactI18next)
.use(resourcesToBackend((language: Locale, namespace: Namespace) => import(`../i18n/${language}/${namespace}.ts`)))
.use(resourcesToBackend((language: Locale, namespace: NamespaceKebabCase) => {
return import(`../i18n/${language}/${namespace}.json`)
}))
.init({
lng: lng === 'zh-Hans' ? 'zh-Hans' : lng,
ns,
defaultNS: ns,
fallbackLng: 'en-US',
keySeparator: false,
})
return i18nInstance
}
export async function getTranslation(lng: Locale, ns: Namespace, options: Record<string, any> = {}) {
export async function getTranslation(lng: Locale, ns: NamespaceKebabCase) {
const camelNs = camelCase(ns) as NamespaceCamelCase
const i18nextInstance = await initI18next(lng, ns)
return {
// @ts-expect-error types mismatch
t: i18nextInstance.getFixedT(lng, ns, options.keyPrefix),
t: i18nextInstance.getFixedT(lng, camelNs),
i18n: i18nextInstance,
}
}