Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1711 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2069. Min Range: -4.2069

Max Act: 4.2069. Min Act: -0.1700

Data Index: 5813216 (Open Web Text)

Max Activating Token Index: 396

Text #1

Max Range: 4.2069. Min Range: -4.2069

Max Act: 4.1080. Min Act: -0.1700

Data Index: 8451074 (Open Web Text)

Max Activating Token Index: 518

Text #2

Max Range: 4.2069. Min Range: -4.2069

Max Act: 4.1160. Min Act: -0.1700

Data Index: 2909547 (Open Web Text)

Max Activating Token Index: 766

Text #3

Max Range: 4.2069. Min Range: -4.2069

Max Act: 4.0303. Min Act: -0.1700

Data Index: 5754020 (Open Web Text)

Max Activating Token Index: 514

Text #4

Max Range: 4.2069. Min Range: -4.2069

Max Act: 4.0452. Min Act: -0.1700

Data Index: 757894 (Open Web Text)

Max Activating Token Index: 877

Text #5

Max Range: 4.2069. Min Range: -4.2069

Max Act: 3.9960. Min Act: -0.1700

Data Index: 1268684 (Open Web Text)

Max Activating Token Index: 388

Text #6

Max Range: 4.2069. Min Range: -4.2069

Max Act: 4.0287. Min Act: -0.1700

Data Index: 4113275 (Open Web Text)

Max Activating Token Index: 58

Text #7

Max Range: 4.2069. Min Range: -4.2069

Max Act: 3.9932. Min Act: -0.1700

Data Index: 7682027 (Open Web Text)

Max Activating Token Index: 933

Text #8

Max Range: 4.2069. Min Range: -4.2069

Max Act: 3.9711. Min Act: -0.1700

Data Index: 1217147 (Open Web Text)

Max Activating Token Index: 116

Text #9

Max Range: 4.2069. Min Range: -4.2069

Max Act: 3.9560. Min Act: -0.1700

Data Index: 4091153 (Open Web Text)

Max Activating Token Index: 613

Text #10

Max Range: 4.2069. Min Range: -4.2069

Max Act: 3.9989. Min Act: -0.1700

Data Index: 36596 (Open Web Text)

Max Activating Token Index: 182

Text #11

Max Range: 4.2069. Min Range: -4.2069

Max Act: 3.9937. Min Act: -0.1700

Data Index: 7681745 (Open Web Text)

Max Activating Token Index: 382

Text #12

Max Range: 4.2069. Min Range: -4.2069

Max Act: 3.9849. Min Act: -0.1700

Data Index: 2109640 (Open Web Text)

Max Activating Token Index: 10

Text #13

Max Range: 4.2069. Min Range: -4.2069

Max Act: 3.9564. Min Act: -0.1700

Data Index: 3028137 (Open Web Text)

Max Activating Token Index: 603

Text #14

Max Range: 4.2069. Min Range: -4.2069

Max Act: 3.9736. Min Act: -0.1700

Data Index: 7617075 (Open Web Text)

Max Activating Token Index: 392

Text #15

Max Range: 4.2069. Min Range: -4.2069

Max Act: 3.9071. Min Act: -0.1700

Data Index: 6569807 (Open Web Text)

Max Activating Token Index: 406

Text #16

Max Range: 4.2069. Min Range: -4.2069

Max Act: 3.9249. Min Act: -0.1700

Data Index: 5477435 (Open Web Text)

Max Activating Token Index: 950

Text #17

Max Range: 4.2069. Min Range: -4.2069

Max Act: 3.8852. Min Act: -0.1700

Data Index: 6549799 (Open Web Text)

Max Activating Token Index: 900

Text #18

Max Range: 4.2069. Min Range: -4.2069

Max Act: 3.9500. Min Act: -0.1700

Data Index: 4343115 (Open Web Text)

Max Activating Token Index: 504

Text #19

Max Range: 4.2069. Min Range: -4.2069

Max Act: 3.8736. Min Act: -0.1700

Data Index: 3541754 (Open Web Text)

Max Activating Token Index: 905