Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2449 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.0432. Min Range: -5.0432

Max Act: 5.0432. Min Act: -0.1700

Data Index: 5355326 (Open Web Text)

Max Activating Token Index: 539

Text #1

Max Range: 5.0432. Min Range: -5.0432

Max Act: 5.0337. Min Act: -0.1700

Data Index: 2914873 (Open Web Text)

Max Activating Token Index: 641

Text #2

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9995. Min Act: -0.1700

Data Index: 8318148 (Open Web Text)

Max Activating Token Index: 715

Text #3

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9862. Min Act: -0.1700

Data Index: 1024315 (Open Web Text)

Max Activating Token Index: 628

Text #4

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9617. Min Act: -0.1700

Data Index: 5771801 (Open Web Text)

Max Activating Token Index: 666

Text #5

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9997. Min Act: -0.1700

Data Index: 2258884 (Open Web Text)

Max Activating Token Index: 527

Text #6

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9357. Min Act: -0.1700

Data Index: 8438375 (Open Web Text)

Max Activating Token Index: 345

Text #7

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9318. Min Act: -0.1700

Data Index: 7703642 (Open Web Text)

Max Activating Token Index: 585

Text #8

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9400. Min Act: -0.1700

Data Index: 4638768 (Open Web Text)

Max Activating Token Index: 201

Text #9

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9292. Min Act: -0.1700

Data Index: 500374 (Open Web Text)

Max Activating Token Index: 495

Text #10

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9296. Min Act: -0.1700

Data Index: 5936552 (Open Web Text)

Max Activating Token Index: 541

Text #11

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9171. Min Act: -0.1700

Data Index: 431193 (Open Web Text)

Max Activating Token Index: 722

Text #12

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9180. Min Act: -0.1700

Data Index: 6733649 (Open Web Text)

Max Activating Token Index: 657

Text #13

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9135. Min Act: -0.1700

Data Index: 6088014 (Open Web Text)

Max Activating Token Index: 125

Text #14

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9310. Min Act: -0.1700

Data Index: 2538948 (Open Web Text)

Max Activating Token Index: 658

Text #15

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9265. Min Act: -0.1700

Data Index: 2260989 (Open Web Text)

Max Activating Token Index: 282

Text #16

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9313. Min Act: -0.1700

Data Index: 5366812 (Open Web Text)

Max Activating Token Index: 133

Text #17

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9197. Min Act: -0.1700

Data Index: 5355327 (Open Web Text)

Max Activating Token Index: 459

Text #18

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.9097. Min Act: -0.1700

Data Index: 4051227 (Open Web Text)

Max Activating Token Index: 200

Text #19

Max Range: 5.0432. Min Range: -5.0432

Max Act: 4.8967. Min Act: -0.1700

Data Index: 1661449 (Open Web Text)

Max Activating Token Index: 636