Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1342 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.2987. Min Act: -0.1700

Data Index: 8566044 (Open Web Text)

Max Activating Token Index: 413

Text #1

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.2634. Min Act: -0.1700

Data Index: 1309829 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.0824. Min Act: -0.1700

Data Index: 5388351 (Open Web Text)

Max Activating Token Index: 427

Text #3

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.2231. Min Act: -0.1700

Data Index: 618391 (Open Web Text)

Max Activating Token Index: 814

Text #4

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.1332. Min Act: -0.1700

Data Index: 3995720 (Open Web Text)

Max Activating Token Index: 304

Text #5

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.0652. Min Act: -0.1700

Data Index: 260478 (Open Web Text)

Max Activating Token Index: 590

Text #6

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.1339. Min Act: -0.1700

Data Index: 5128053 (Open Web Text)

Max Activating Token Index: 752

Text #7

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.0803. Min Act: -0.1700

Data Index: 1022821 (Open Web Text)

Max Activating Token Index: 629

Text #8

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.1136. Min Act: -0.1700

Data Index: 5709704 (Open Web Text)

Max Activating Token Index: 688

Text #9

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.1039. Min Act: -0.1700

Data Index: 7179777 (Open Web Text)

Max Activating Token Index: 666

Text #10

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.0931. Min Act: -0.1700

Data Index: 546689 (Open Web Text)

Max Activating Token Index: 525

Text #11

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.0491. Min Act: -0.1700

Data Index: 3140211 (Open Web Text)

Max Activating Token Index: 390

Text #12

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.1106. Min Act: -0.1700

Data Index: 67815 (Open Web Text)

Max Activating Token Index: 3

Text #13

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.0377. Min Act: -0.1700

Data Index: 1195291 (Open Web Text)

Max Activating Token Index: 612

Text #14

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.0862. Min Act: -0.1700

Data Index: 5752965 (Open Web Text)

Max Activating Token Index: 621

Text #15

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.0563. Min Act: -0.1700

Data Index: 2277418 (Open Web Text)

Max Activating Token Index: 481

Text #16

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.0307. Min Act: -0.1700

Data Index: 1366968 (Open Web Text)

Max Activating Token Index: 926

Text #17

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.0657. Min Act: -0.1700

Data Index: 1860071 (Open Web Text)

Max Activating Token Index: 110

Text #18

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.0216. Min Act: -0.1700

Data Index: 965694 (Open Web Text)

Max Activating Token Index: 454

Text #19

Max Range: 4.2987. Min Range: -4.2987

Max Act: 4.0366. Min Act: -0.1700

Data Index: 5823815 (Open Web Text)

Max Activating Token Index: 547