Machine translation and human evaluation of translated texts

[mrmvp]

I want to conduct a research on translation using machine translation from a language to English. How can I evaluate the translation in English in terms of readability, fluency, and accuracy in human evaluation and non machine evaluation. How can I judge the quality of the text generated by machine translation? Are there websites to evaluate the translated texts?

 

[jutfrank]

I think you're saying you want a human to do it but you can't do it yourself, right?

 

[mrmvp]

I think you're saying you want a human to do it but you can't do it yourself, right?

Yes, the problem is that I am not an expert in Arabic (even though it is my native language) and do not have Arabic teaching qualifications to evaluate translated texts. The purpose of this thread is to explore ways to evaluate translation based on human assessment. I will ask my colleagues at the University to help me evaluate the texts generated by machine translation.
Should I ask something like: "On a scale of 1 to 5, how readable is the following text?"

1
2
3
4
5

Are there websites that can help me evaluate text readability, fluency, and accuracy?

I have to ask both esteemed English teachers at this lovely community Using English and also seek help from statisticians.

 

[jutfrank]

Yes, the problem is that I am not an expert in Arabic (even though it is my native language) and do not have Arabic teaching qualifications to evaluate translated texts.

Is the goal then to evaluate translation from English into Arabic? Is that specifically what you aim to evaluate?

The purpose of this thread is to explore ways to evaluate translation based on human assessment. I will ask my colleagues at the University to help me evaluate the texts generated by machine translation.

Yes. If you require a human to do it, then you have to find people who are expert Arabic users but who also have a very high proficiency in English, as they'd need to be able to understand the original English. You won't find many people who fit that bill. You really should get a machine to do this for you.

Should I ask something like: "On a scale of 1 to 5, how readable is the following text?"

What do you mean by 'readable'? Do you mean 'natural'?

Are there websites that can help me evaluate text readability, fluency, and accuracy?

Yes, of course, but it's not people who do this. It's machines.

I have to ask both esteemed English teachers at this lovely community Using English and also seek help from statisticians.

We can evaluate English but we can't judge the Arabic for you.

Why are you doing this? Are you trying to develop your own translation tool?

 

[mrmvp]

Is the goal then to evaluate translation from English into Arabic? Is that specifically what you aim to evaluate?

Yes

Yes. If you require a human to do it, then you have to find people who are expert Arabic users but who also have a very high proficiency in English, as they'd need to be able to understand the original English. You won't find many people who fit that bill. You really should get a machine to do this for you.

I should not tell the Arabic teachers that they are going to evaluate texts that had been translated so they might have prejudgment on the texts before translation. Arabic teachers who have good command of Arabic can easily do Error analysis and evaluate the texts upon that.

What do you mean by 'readable'? Do you mean 'natural'?

Yes. I will ask them to evaluate the accuracy and fluency too . They will rate from 1 to 5

Yes, of course, but it's not people who do this. It's machines.

I will combine both machine translation and human evaluation. There are some metrics like BULE, F1 score and logical expression metrics. I don't know if they are websites to evaluate Arabic texts readability, fluency and accuracy?

We can evaluate English but we can't judge the Arabic for you.

Why are you doing this? Are you trying to develop your own translation tool?

Please check the other thread. Thank you so much.