Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2831 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.7226. Min Act: -0.1700

Data Index: 2566198 (Open Web Text)

Max Activating Token Index: 471

Text #1

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.7526. Min Act: -0.1700

Data Index: 6401430 (Open Web Text)

Max Activating Token Index: 752

Text #2

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.6732. Min Act: -0.1700

Data Index: 4034027 (Open Web Text)

Max Activating Token Index: 365

Text #3

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.6226. Min Act: -0.1700

Data Index: 5023667 (Open Web Text)

Max Activating Token Index: 905

Text #4

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.6682. Min Act: -0.1700

Data Index: 1883205 (Open Web Text)

Max Activating Token Index: 472

Text #5

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.6491. Min Act: -0.1700

Data Index: 5370994 (Open Web Text)

Max Activating Token Index: 50

Text #6

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.6176. Min Act: -0.1700

Data Index: 1697270 (Open Web Text)

Max Activating Token Index: 950

Text #7

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.6155. Min Act: -0.1700

Data Index: 6386399 (Open Web Text)

Max Activating Token Index: 633

Text #8

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.6369. Min Act: -0.1700

Data Index: 3137458 (Open Web Text)

Max Activating Token Index: 792

Text #9

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.6249. Min Act: -0.1700

Data Index: 3505205 (Open Web Text)

Max Activating Token Index: 848

Text #10

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.5710. Min Act: -0.1700

Data Index: 8088250 (Open Web Text)

Max Activating Token Index: 437

Text #11

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.5707. Min Act: -0.1700

Data Index: 7483564 (Open Web Text)

Max Activating Token Index: 259

Text #12

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.5860. Min Act: -0.1700

Data Index: 3190885 (Open Web Text)

Max Activating Token Index: 444

Text #13

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.5868. Min Act: -0.1700

Data Index: 7353618 (Open Web Text)

Max Activating Token Index: 143

Text #14

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.6288. Min Act: -0.1700

Data Index: 6999882 (Open Web Text)

Max Activating Token Index: 299

Text #15

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.5871. Min Act: -0.1700

Data Index: 5656099 (Open Web Text)

Max Activating Token Index: 464

Text #16

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.5558. Min Act: -0.1700

Data Index: 2221087 (Open Web Text)

Max Activating Token Index: 392

Text #17

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.5507. Min Act: -0.1700

Data Index: 1871632 (Open Web Text)

Max Activating Token Index: 483

Text #18

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.5619. Min Act: -0.1700

Data Index: 5882010 (Open Web Text)

Max Activating Token Index: 317

Text #19

Max Range: 5.7526. Min Range: -5.7526

Max Act: 5.5561. Min Act: -0.1700

Data Index: 4666068 (Open Web Text)

Max Activating Token Index: 210