Implementation Details. The Implementation design as shown in Figure 6 follows the three layer design pattern containing data layer, business logic layer and presentation layer. The main system entities are implemented using the following technologies: • Custom web parts and web pages: In the presentation layer, custom web parts and web pages are used to provide an easy to use portal for customers to add or update their requests. •
Implementation Details. In order to implement the cooperation protocol we modified the wireless card driver and the DHCP client. Furthermore, a
Implementation Details. Model Configuration The Transformer architec- ture we use is composed of 24 encoder layers and 12 interleaved decoder layers. Furthermore, the architecture has an embedding size of 1024, with a dropout rate of 0.1. The feed-forward network has a size of 4096, with 16 attention heads. For parameter initialization, we follow Ma et al. (2021) and Xxxx et al. (2021b) to pre-train a strong MT system with sentence-level bilingual data. For the rest of this paper, We call it the Baseline Model and use it as a strong pre-trained baseline system. Data Pre-processing For all of the experiments conducted in this paper, we use SentencePiece (Kudo and Xxxxxxxxxx, 2018) for tokenization. The SentencePiece model we use is the same as Yang et al. (2021b). Also, we follow prior works to prefix the source input translation texts with a language tag that indicates the target language of the outputs. Evaluations We use the BLEU scores (Xxxxxxxx et al., 2002) computed with the script from Sacre- BLEU for evaluation.5 Training Details We use the Xxxx optimizer (Xxxxxx and Ba, 2014) and set it with the hyperpa- rameter β1 = 0.9 and β2 = 0.98 for downstream fine-tuning. We set the learning rate as 1e-5, with a warmup step of 4000. We use the label smooth- ing cross-entropy for the standard translation loss and we set label smoothing with a ratio of 0.1 for model training. All of the fine-tuning exper- iments reported in this paper are conducted on 8 4Empirically, we have found that setting a balanced value with α = 0.5 brings a good performance. 5xxxxx://xxxxxx.xxx/mjpost/sacrebleu Model Fr→En De→En Zh→En Vi→En Cs→En Th→En Avg. Sentence-level Systems HAN† (Miculicich et al., 2018) M2M-100 (Fan et al., 2022) - 50.18 - 42.24 24.00 26.62 - 34.92 - 37.84 - 27.28 - 36.51 xXXXX (Xxx et al., 2020) 48.69 44.80 28.39 37.18 39.47 - - Baseline Model + BT 50.69 47.07 30.35 39.59 43.05 32.30 40.51 Document-level Systems mT5† (Xxx et al., 2021a) M2M-100 (Fan et al., 2022) - 49.43 - 43.82 24.24 26.63 - 35.91 - 39.04 - 25.93 - 36.79 xXXXX (Xxx et al., 2020) 49.16 44.86 29.60 37.09 39.64 - - XXXXX† (Xxxxx et al., 2020) Baseline Model + BT - 49.53 - 45.98 28.40 30.17 - 39.28 - 42.33 - 30.62 - 39.65 Baseline Model + BT + MA (Xxxx et al., 2021c) 48.99 47.34 30.35 39.79 43.01 32.14 40.27 Systems with Bilingual Parallel Document Data for Pre-training DOCmT5† - - 31.40∗ - - - - BMA-SBT + BT 51.10 47.59 30.80 40.20 43.17 32.23 40.85 Ablation Study - w/o KL1 49.58 46.38 29.46 39.09 42.87 30.59 39.66 - w/...
