incorrect symbols in terminal

[kancler]

Hello, guys! Can you help me with a problem, please?
I use C on Ubuntu.
When I try to print a single Cyrillic symbol with '\n' (or any other symbol) at the end I get '?' in terminal, with ASCII symbols all is right.

char buff[255] = "хай, man";

    for (int i=0; i<(strlen(buff)); i++) {
         printf("%c\n", buff[i]);
     }

If I print Cyrillic symbols in a raw (without '\n') - I get ? only at the end of the string.

 

[Deleted member 2829]

Your C compiler probably uses UTF8 for character encoding. This means that ASCII characters are one byte and Cyrillic characters are two bytes. So you can't loop through them as if they were all one byte ! You'll need to read up on handling Unicode characters. It's not an easy subject.

 

[kancler]

Your C compiler probably uses UTF8 for character encoding. This means that ASCII characters are one byte and Cyrillic characters are two bytes. So you can't loop through them as if they were all one byte ! You'll need to read up on handling Unicode characters. It's not an easy subject.

Thank you! Now it is clear where is problem.

 

[Deleted member 2829]

Thank you! Now it is clear where is problem.

Yes ! I've seen and know about this stuff, but I've never programmed it myself. So I can't give you a ready solution, sorry. If you get it to work do let us know !

 

[kancler]

Yes ! I've seen and know about this stuff, but I've never programmed it myself. So I can't give you a ready solution, sorry. If you get it to work do let us know !

Helped me solve this issue and I figured out a little how to work with unicode.

#include <ctype.h>
#include <stdlib.h>
#include <stdio.h>

int main(void) {
  char buff[255] = u8"хай, man";
  size_t len = strlen(buff);

  for (size_t i = 0; i<len; i++) {
    char ch = buff[i];
    // If ASCII character ....
    if ((ch & 0x80) == 0) { 
      printf("%c\n", ch);
    } else {
      // Process UTF-8
      char b[5] = { ch };
      size_t j;
      for (j = 1; (j < 4) && ((buff[i+j] & 0xC0) == 0x80); j++) {
        b[j] = buff[i+j];
      }
      b[j] = 0;
      printf("%s\n", b);  // Print 1 UTF-8 character.
      i += j - 1;
    }
  }
  return 0;
}

 

[Deleted member 2829]

Great that you got it working ! I'm not sure this is the best way to go about it though. You are still treating your string as an array of 8-bit characters, rather than as an array of Unicode characters. The idea is to use a suitable datatype rather than char, and use functions that work on that datatype without having to worry about the encoding. I know this may sound a bit vague but I hope you get the general idea. I'm sure there are good tutorials for this.

 

[kancler]

Great that you got it working ! I'm not sure this is the best way to go about it though. You are still treating your string as an array of 8-bit characters, rather than as an array of Unicode characters. The idea is to use a suitable datatype rather than char, and use functions that work on that datatype without having to worry about the encoding. I know this may sound a bit vague but I hope you get the general idea. I'm sure there are good tutorials for this.

Yes, I heard about wchar_t and similar types but I haven't worked with them yet, maybe later 🙂 Thanks for your attention and tips!