Gabe Newell: Yes, We're Still Working on Half-Life 2: Episode 3

[Jumplion]

Strictly speaking, programing is justing learning another language, where you have to have %100 correct grammar to get anything done. Just like most languages around the world, it has alot to do with acclimation (People native to Iceland won't find Icelandic hard.) And a degree of objective complexity. (Icelandic is obscenely complicated, commonly considered the most difficult language in the world.)

The objective standpoint is more prominent in coding, where specific structure, syntax, ect is not just a nuisance but vital. Or having needlessly esoteric, sloppy, and lengthy structure increases chances of failure.

but,

I plainly have no idea what the PS3 would be like to code for. If it's just different, as I personally think mr. newell comments indicate, then it would be a similar conundrum to how it's generally much harder for a person who speaks english to learn an eastern language like Japanese, than it would be to learn a similar romantic language like german.

I don't know what the PS3's like of course, but I thought I'd clarify that point of programming, since it seems to be so contentious in these discussions.

I have no doubt that coming from a primarily PC developer, VALVe would have trouble programing for a completely different platform such as the PS3. If they're really trying to work with it, they can't expect to be gods at it overnight, it will take time. It's just like learning a new language, it may be difficult and different (those things usually come together) but if you keep at it it'll be fine.

 

[akmarksman]

Im gonna buy L4D2 because I found the first one fun and I don't care that much about HL or any episodes of it.

 

[cleverlymadeup]

Your also clearly not a programmer. Something that is difficult to program for even after you know how to program it will always be difficult unless you come up with some tricks. Saying that something isn't difficult its just different isn't really true when it comes to programming. It's pretty much almost all the time either easier or its more difficult.

actually you are NOT a programmer either, if you are then you are a rather bad one.

i DO happen to know some coding and have done it before. the analogy in here of it being like a spoken language is correct.

first off much like spoken languages there are families of computer languages, they aren't identical but they do tend to share a bunch of similarities. the main one being the family of C, which includes such relatives as C, C++, Java, C#, C++0x and a few others. they aren't the same language but going from one to the other isn't that hard and pretty easy for anyone good at coding to pick up

secondly if you actually learn the language it will become and easier to code for. it you are still finding it difficult after a year or more of coding with it, then well maybe you aren't as good at coding as you think you are

really you should learn a bit about computer languages before you go saying stuff about them, especially on a forum where people have probly forgotten more about coding than you know

 

[Zer_]

This has nothing to do with programming language... at all.

People who say the PS3 gives them a lot of freedom are outright lying. Especially when it comes to programming. To use the Cell to it's full potential you not only have to multi-thread your entire code, but you must also vectorize it...

Vector (or SIMD) programming is applicable to a very wide range of problems, not just media codecs and the like. Searching, pattern matching, networking and even OS functions such as memory operations can be accelerated. The performance difference is not small either, Altivec programmers regularly talk of factors of 3x to 16x better performance in functions they?ve modified.

The SPEs have a reputation of being weird DSP type things which can only handle vector code. This is not true as they are also quite capable of handling scalar (read: normal) code. It?s better to use vector code however because processing 4 (or more) pieces of data at a time is obviously going to be faster.

SPEs can be programmed in assembly if you feel adventurous enough but it?s not necessary, it is normal to program the SPEs in C, C doesn?t have direct support for the SPE instruction set so you use a series of additional commands called ?intrinsics?.

To vectorise an algorithm it will need to be restructured so it can deal with multiple elements in one go.

This can be pretty easy:

for (count = 0; count < 100; count++)

{

C[count] = A[count] + B[count]

}

Vectorises as:

for (count = 0; count < 25 ; count++)

{

VC[count] = spu_add(VA[count], VB[count])

}

The ?spu_add? function is the intrinsic you use for add on an SPE.

For vector processing data should be aligned properly and have a regular access pattern. An array is obviously ideal, (?vector processing? literally means ?array processing?).

If your data is in an array but not properly aligned you can use the shuffle instruction to fix this. Shuffle commands allow you If you want, to manipulate two unaligned arrays and write back the results to another unaligned array. Only a few shuffle shuffle commands are necessary per iteration to do this, obviously it?s better to use aligned data as you don?t need extra operations but it?s useful nevertheless.

If you?ve ever programmed MMX / SSE or AltiVec you?ll immediately feel right at home working on an SPE as you use many of the same techniques, AltiVec in particular is very similar. It should be noted however that the SPE instruction set and AltiVec while very similar, are not identical. There are different instructions, different data types, the SPEs have more registers and of course the SPEs have a local store.

Furthermore, you as the programmer must manage the memory for each SPE...

Each SPE contains a small fast memory called a ?local store?. The local stores are not the same as conventional CPU caches because a cache is pretty much invisible to programmers and it does it?s thing in the background, you do not normally use the cache directly unless you make use of cache pre-fetch commands.

The local store is visible to Cell developers and you have no choice but to deal with it (at least until advanced compilers come along which can hide it). The SPEs use the local store for holding their data and programs, as they cannot directly access main memory. The SPEs can however move data to or from the rest of the system via DMA (Direct Memory Access) commands which usually move 16 bytes (or multiples thereof) up to 16 Kilobytes in one go. DMA commands can move less (as little as 1 byte) but this is best avoided as it?s very inefficient.

Manual control of the local store may sound like a royal pain but it?s very useful since you can use it at times to completely hide memory access, something conventional processors have difficulty with even with large caches and aggressive pre-fetching engines. If you can mange to hide memory accesses the SPE will keep processing data without interruption. For example: by hiding 99% of the memory access, IBM managed to get a 50x speedup compared to a 2GHz G5 in their terrain renderer.

You hide the memory access by using the common techniques of double or multi-buffering. You DMA in a block of data (lets call it A) and start processing it, while data A is being processed you DMA in a second block (B) of data. Once data A is processed the SPE starts processing block B, while this is happening the results of A are DMA?d back to memory and a third block of data, C is read into where block A was sitting. Once B is completed the SPE starts processing C and the process repeats. Multi-buffering is much the same as double buffering but you use more than 2 buffers.

Double buffering is a very common technique so there?s nothing magic about it. It is not typically used to read data into traditional processors though as there are no buffers to read data into, you can try using cache but it?s not quite the same thing as it?s not addressable and can be flushed.

It may sound like it will only work with array type of data but it can also be used on certain types of tree structures so it?s not as limited as it might seem.

Current PS3 (Cell) compilers will not hide the memory management for the programmer, which essentially means that it is completely up to the programmer to manage the cache for each separate SPE.

I think the ultimate answer to the question will be dependent on the individual programmer.

-If you are used to programming with high level languages such as Perl or Python this will all seem rather alien to you and you?ll be more likely to get the impression Cell development will be difficult.

-If you?ve developed optimized C or C++ you should have a better understanding of what?s involved and will recognize that extra work is involved.

-If you?ve developed SSE and / or Altivec code in C and understand multithreading you?ll probably recognize almost everything here and be wondering what all the fuss is about.

-If you?ve developed for the Playstation 2 or tried general purpose programming on GPUs you may be thinking Cell development looks easy!

As I've previously stated (in a post a few months back) the Cell is a surprisingly simple piece of hardware. Today's processors have highly advanced features like MMX that will help manage the workload throughout the entire processor. The Cell is a much simpler design then say a standard Core 2 Duo in that it does not have all the necessary circuits to manage internal memory and other such things. These things must all be taken into consideration when programming for the Cell. The only exception when it comes to the Cell is the PPE, although even the PPE is a simplified PowerPC core.

The idea behind the Cell was to provide a simple processor that would run cool and run fast. The idea works well. Ultimately I think with the current trend of AMD and Intel processors running cooler while using less energy means that the Cell's approach to the issue isn't all that good for programmers. The Cell works well for tasks like number crunching and scientific applications, but for the gaming industry the focus needs to be increasing the efficiency of developing and making games cheaper to make. I fully understand Valve's stance on the PS3 and I agree with their reasons.

 

[Kstreitenfeld]

Personally, I'd prefer a sequal to Portal then one to counter strike. I much prefer more brilliant story, humour and puzzles than just a few different ways to shoot terrorists.

Agreed. Single player games get more out of sequels then multi-player shooters.

 

[Weaver]

Episode 3 had better be like 40 hours long, God damn.

 

[Noliving]

actually you are NOT a programmer either, if you are then you are a rather bad one.

i DO happen to know some coding and have done it before. the analogy in here of it being like a spoken language is correct.

first off much like spoken languages there are families of computer languages, they aren't identical but they do tend to share a bunch of similarities. the main one being the family of C, which includes such relatives as C, C++, Java, C#, C++0x and a few others. they aren't the same language but going from one to the other isn't that hard and pretty easy for anyone good at coding to pick up

secondly if you actually learn the language it will become and easier to code for. it you are still finding it difficult after a year or more of coding with it, then well maybe you aren't as good at coding as you think you are

really you should learn a bit about computer languages before you go saying stuff about them, especially on a forum where people have probly forgotten more about coding than you know

No I'm a programmer but a rather bad one as I'm still just begining. It is very much like a spoken language, however though to say something is just different and can't be more difficult or easier no matter how much experience you have with the languages computer or spoken is false.

For example memory management is pretty much easier in java then in C++, not always of course, sure you could be perfect in C++ and be able to code it in less 10 seconds with no errors and be able to do it every time but doing memory management in java vs. C++ will probably always be easier no matter how much experience you have, unless they change it in either C++ or java. That is the point. Of course C++ has advantage over java.

Here is another example to make my point. The hello world program.

Lets look at the first known version of the hello world program in C:

int main()
{
printf("hello, world");
return 0;
}


Now lets look at the version of the hello world program in computer language B:

main( ) {
extrn a, b, c;
putchar(a); putchar(b); putchar(c); putchar('!*n');
}
a 'hell';
b 'o, w';
c 'orld';


As you can see no matter how much experience you have with the computer language B, doing the hello world program in C will always be easier, it won't just be different but easier all the time.

 

[Noliving]

I'll try my best to find some quotes, but I am sure that there are many developers who say

Well I'm still waiting.

So if I, someone who has not programed beyond BB Code, start learning C++ or something, it will still be hard for me 2 years of active programing? If that's the case, we need to rework how we code.

Clearly I obvious didn't explain what I meant by that. What I meant was sure you could learn the language and be able to do it on your first time everytime before compiling and not have any errors but it will always be more work to do it in this language vs another language that is what I meant. For example memory management in java will pretty much always be easier then in C++, sure you could be an expert at C++ but you will probably end up finding that it will be less work to do it in java then it is to do in C++, C++ though of course has advantages over java.

Another example would be doing the hello world program in C vs. B, its less work to do the hello world program in C then it is in B, sure the hello world program is easy but you will probably find that doing it in C will always be easier then doing it in B because it requires less work.

Saying that something isn't difficult its just different isn't really true when it comes to programming.

So, it's true for pretty much everything else in life (Riding a bike, learning to spell, speak, ect...) but it's not true in programing?

Think of it this way, is it easier all the time to walk through a door that automatically opens for you like at a store or is it easier to up to a door that has a knob that requires you to turn it or push and then you have to either push or pull the door to open and then walk through. Sure you could probably turn the door knob and either push or pull the door and walkthrough it in your sleep but it will always be easier to walkthrough a door that opens automatically for you. I mean you can't say its just different and that they are both just as easy or difficult, they are not, one is more difficult then the other. Sure you can get to a point with the more difficult one where it isn't a pain in the butt anymore but compared to the other one it will be more difficult.

Also I think your underestimating how much time valve has actually spent looking at the white papers for the ps3.

And pray, do tell, how much time has VALVe spent looking on the white papers of the PS3? Papers aren't hardware last I checked.

Also, I am greatly honored that I am your first post

you are correct White papers are not hardware, do you know why whitepapers are important?

Here is an exmaple of an intel one: http://cache-www.intel.com/cd/00/00/40/65/406592_406592.pdf

Here is another intel one that helps developers optimize their code for IA-32 processors:

http://software.intel.com/sites/products/collateral/hpc/compilers/optimizing_applications_with_intel_compilers.pdf

As you can see whitepapers for example can help explain how to write code for hardware.

My guess is that valve along with all software developers have spent quite a bit of time looking at the white papers for ps3.

 

[Jumplion]

Well I'm still waiting.

Alright, I'll try my best to search for quotes, just be patient I'm not the best at searchings. I'll admit my wrongings if I have to, I'm not afraid to lose.

So if I, someone who has not programed beyond BB Code, start learning C++ or something, it will still be hard for me 2 years of active programing? If that's the case, we need to rework how we code.

Clearly I obvious didn't explain what I meant by that. What I meant was sure you could learn the language and be able to do it on your first time everytime before compiling and not have any errors but it will always be more work to do it in this language vs another language that is what I meant.

But.....wah? So, I could be perfect at a language, do it right every time, but it will always be more work in that language versus another language that is harder? I do not get what you're saying, are you saying that no matter how much I code it will always be harder to work for regardless of my experience?

If that's the case, there's a problem.

For example memory management in java will pretty much always be easier then in C++, sure you could be an expert at C++ but you will probably end up finding that it will be less work to do it in java then it is to do in C++, C++ though of course has advantages over java.

You're missing my point with this, because C++ is so similar to Java (or at least, you say it is, I'll take your word for it) you can't exactly compare the two as completely different because they're in the same damn family. To my knowledge, PS3 architecture is completely different from "regular" coding like C++ and the like. It's understandable that a company that centers around a specific platform would have trouble developing for a brand new platform.

Another example would be doing the hello world program in C vs. B, its less work to do the hello world program in C then it is in B, sure the hello world program is easy but you will probably find that doing it in C will always be easier then doing it in B because it requires less work.

Again, this fails because according to you these are nearly the same program with different perks. We're not talking about one version being better to use than the other, we're talking about two completely different programs. Are you telling me that if I work with C++ for most of my life, and work with, I dunno, some other completely random code program or something for the other half of my life, I will forever be bamboozled by the complexity of the second program?

Think of it this way, is it easier all the time to walk through a door that automatically opens for you like at a store or is it easier to up to a door that has a knob that requires you to turn it or push and then you have to either push or pull the door to open and then walk through. Sure you could probably turn the door knob and either push or pull the door and walkthrough it in your sleep but it will always be easier to walkthrough a door that opens automatically for you. I mean you can't say its just different and that they are both just as easy or difficult, they are not, one is more difficult then the other. Sure you can get to a point with the more difficult one where it isn't a pain in the butt anymore but compared to the other one it will be more difficult.

I kind of get what you're trying to say, but you don't learn to open doors. This falls under the "too similar to be "completely different"" thing, I suppose my analogy also failed in that.

I've ridden a skate board my entire life, I live my life on four wheels. Then I need to start learning how to ride a bike, I think we can agree that a skate board and a bike are two different things. Riding a bike is completely different for me, I've never rode one in my life. It's understandable that I would have trouble learning how to use this completely different vehicle. But if I keep at it, I will eventually learn, and maybe I'll even be a pro or something. Does that make more sense?

you are correct White papers are not hardware, do you know why whitepapers are important?

Here is an exmaple of an intel one: http://cache-www.intel.com/cd/00/00/40/65/406592_406592.pdf

Here is another intel one that helps developers optimize their code for IA-32 processors:

http://software.intel.com/sites/products/collateral/hpc/compilers/optimizing_applications_with_intel_compilers.pdf

As you can see whitepapers for example can help explain how to write code for hardware.

My guess is that valve along with all software developers have spent quite a bit of time looking at the white papers for ps3.

So? You have no proof otherwise, all you're saying is "Oh, VALVe could be looking at the papers!" when you have no proof that they are or aren't. You're assuming they are to help your point, and I ask you to not do that. Even if they were, looking at papers is not the same as actively programing for it. VALVe has done little to nothing with the PS3, they gave EA to port the Orange Box, the only real thing they've done on a Sony platform is Half-Life on the Ps2.

 

[Noliving]

But.....wah? So, I could be perfect at a language, do it right every time, but it will always be more work in that language versus another language that is harder? I do not get what you're saying, are you saying that no matter how much I code it will always be harder to work for regardless of my experience?

If that's the case, there's a problem.

Well it wouldn't be so hard for you to understand if you didn't leave out the example, that is why I included an example which was the memory management. Ya I am saying that, for example memory management. Java basically requires zero memory management, C++ on the otherhand, depending upon what your coding can require extensive memory management. No matter how good you are at memory management in C++ will never be able to catch up to someone coding in java in terms of memory management because java doesn't really require you to do memory management. Here is another example that has nothing to do with programming, which one would have an easier time getting to 60mph, a bike rider or a car? Lets say your a professional bike rider, the best one in the world and I have just started driving a car, no matter how good you are at riding your bike I'm always going to have an easier time at getting to 60mph then you are in your bike.

Lets say you have equal skill in java vs. C++ and you want to code "hello world program" Guess which one is easier? C++ is, you know why? Because the C++ basically requires less lines of code and less characters in those lines of code.

Here is another example

Which one is easier: int bar(int i)(C++) or public int bar(int i)(java)

The first one is because it requires one less word. So matter how good you are at java the C++ will always be easier because it requires one less word. Ya you could be an expert at java and it could come as second nature writing that line of code out but the C++ will always be easier because again it requires one less word.

You're missing my point with this, because C++ is so similar to Java (or at least, you say it is, I'll take your word for it) you can't exactly compare the two as completely different because they're in the same damn family. To my knowledge, PS3 architecture is completely different from "regular" coding like C++ and the like. It's understandable that a company that centers around a specific platform would have trouble developing for a brand new platform.

Java and C++ have very similar syntax but they have enough differences. I know what your point is and I disagree with your point that it isn't more difficult to program for one platform vs. another completely different platform if you have the same skill level in both platforms, it's just different. That just isn't true, because one will always have parts that are more difficult to write then the other. One will either always require more lines of code then the other or more memory management or more characters in a line of code, etc.

Again, this fails because according to you these are nearly the same program with different perks. We're not talking about one version being better to use than the other, we're talking about two completely different programs. Are you telling me that if I work with C++ for most of my life, and work with, I dunno, some other completely random code program or something for the other half of my life, I will forever be bamboozled by the complexity of the second program?

Actually it does work because we are talking about a game going across multiple platforms because valve doesn't do exclusives anymore(except maybe dod). You honestly think valve would announce a new game and have it only exclusive to the pc or the ps3 or the 360? So no we are not talking about different programs because valve doesn't do exclusives.


You have to be more specific on what the random code program is because programming doesn't work that way. No I never said you would forever be bamboozled by the complexity of the program, I'm saying one would take more effort then the other, so in a sense it would be more difficult, difficult does not mean it is hard, something that could be more difficult then something else could also be incredibly easy, again the door example, the door knob is more difficult then the door that automatically open but that doesn't mean the door knob is hard to do.

I kind of get what you're trying to say, but you don't learn to open doors. This falls under the "too similar to be "completely different"" thing, I suppose my analogy also failed in that.

Yes you do learn how to open doors just like how you learn to lock and unlock a door

I've ridden a skate board my entire life, I live my life on four wheels. Then I need to start learning how to ride a bike, I think we can agree that a skate board and a bike are two different things. Riding a bike is completely different for me, I've never rode one in my life. It's understandable that I would have trouble learning how to use this completely different vehicle. But if I keep at it, I will eventually learn, and maybe I'll even be a pro or something. Does that make more sense?

Yes your going by individual skill and I'm not. For example which one is more likely to tip over? A bike or a skateboard, sure you could be good at a bike and never tip over but a skateboard will always be less likely to tip over.

So? You have no proof otherwise, all you're saying is "Oh, VALVe could be looking at the papers!" when you have no proof that they are or aren't. You're assuming they are to help your point, and I ask you to not do that. Even if they were, looking at papers is not the same as actively programing for it. VALVe has done little to nothing with the PS3, they gave EA to port the Orange Box, the only real thing they've done on a Sony platform is Half-Life on the Ps2.

I don't think I really need proof because in this case its really more of a given because companies such as sony would use white papers as a way to show off all the pros to the system to developers. Your right its not like they are actively programming it but then again white papers that deal with interacting with the hardware generally contain code examples showing you how to interact with the system and if you look at the code and read the code you start to get an idea of what is required. If your a software developer one of the first things you look at when it comes to new software and or hardware is white papers because a lot of them are tutorials.

 

[Jumplion]

Condense

Well, what ev's, we'll agree to disagree I suppose. I'll take my stance, you take yours. Deal?

 

[Noliving]

Deal.

 

[Baconator96]

fuck you, valve. i bought the half-life series, i bought counter-strike, i even bought the goddamn orange box, and now youre punishing the people that didnt have the clairvoyance to buy the right console? fucking two timing bastards

 

[cleverlymadeup]

No I'm a programmer but a rather bad one as I'm still just begining. It is very much like a spoken language, however though to say something is just different and can't be more difficult or easier no matter how much experience you have with the languages computer or spoken is false.

For example memory management is pretty much easier in java then in C++, not always of course, sure you could be perfect in C++ and be able to code it in less 10 seconds with no errors and be able to do it every time but doing memory management in java vs. C++ will probably always be easier no matter how much experience you have, unless they change it in either C++ or java. That is the point. Of course C++ has advantage over java.

wrong again, one of my friends who went to university for comp sci at one of the best schools in the world for it, hint they are one of the few to have their own unix distro, he had never ever tried Java but was fairly good with C and C++. he took to Java with no problems at all because he had experience with C and C++ and Java has a lot of basis in those two languages

Here is another example to make my point. The hello world program.

Lets look at the first known version of the hello world program in C:

int main()
{
printf("hello, world");
return 0;
}


Now lets look at the version of the hello world program in computer language B:

main( ) {
extrn a, b, c;
putchar(a); putchar(b); putchar(c); putchar('!*n');
}
a 'hell';
b 'o, w';
c 'orld';


As you can see no matter how much experience you have with the computer language B, doing the hello world program in C will always be easier, it won't just be different but easier all the time.

well not withstanding the fact that B was used before C and designed for VERY different systems. they aren't even the same type of language.

doing the hello world program in VB or strict BASIC or perl or python is a lot easier than doing it in C, that doesn't prove your point, it just means those languages have different ways of handling certain things

the main thing is knowing one that is part of the same family and syntax makes learning the other a lot easier

 

[Crayzor]

I thought the point of episodic content was to reduce the wait between installments, not lengthen them.

 

[matt87_50]

honestly, has valve ever done a game that wasn't for Windows PC? when proper console devs are struggling with the PS3, I can imagine why valve would be afraid of it. the xbox on the other hand is basically a PC.

 

[Diablini]

Pahahaha! When I saw the tittle I Loled! Yes, We're still working on it! Haha!

 

[jibjab963]

Hes joking right? I don't remember a single major update for the orange box on the 360. And if there has been please tell me what it is.

 

[Fraught]

Portal 2 and Counter-Strike 2? What's there to choose?

 

[internetzealot1]

It's pretty obvious why they don't develop for the Ps3... just look at sales data for the consoles, and you'll see that it's way more efficient to develop exclusively for 360. This is probably the same reason SS:Conviction is exclusive, why the 360 is getting 2 exclusive map packs for MW2, and why FF XIII and MGS are coming to the 360.