Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2856 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0055. Min Range: -4.0055

Max Act: 4.0055. Min Act: -0.1700

Data Index: 5665751 (Open Web Text)

Max Activating Token Index: 710

Text #1

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.9868. Min Act: -0.1700

Data Index: 5221760 (Open Web Text)

Max Activating Token Index: 995

Text #2

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.9447. Min Act: -0.1700

Data Index: 4583056 (Open Web Text)

Max Activating Token Index: 626

Text #3

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.9737. Min Act: -0.1700

Data Index: 4517608 (Open Web Text)

Max Activating Token Index: 406

Text #4

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.9228. Min Act: -0.1700

Data Index: 2070839 (Open Web Text)

Max Activating Token Index: 603

Text #5

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.9411. Min Act: -0.1700

Data Index: 7741402 (Open Web Text)

Max Activating Token Index: 717

Text #6

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.9493. Min Act: -0.1700

Data Index: 1430735 (Open Web Text)

Max Activating Token Index: 530

Text #7

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.9377. Min Act: -0.1700

Data Index: 3194273 (Open Web Text)

Max Activating Token Index: 779

Text #8

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.8927. Min Act: -0.1700

Data Index: 1019086 (Open Web Text)

Max Activating Token Index: 77

Text #9

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.9200. Min Act: -0.1700

Data Index: 5077882 (Open Web Text)

Max Activating Token Index: 273

Text #10

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.8952. Min Act: -0.1700

Data Index: 3264572 (Open Web Text)

Max Activating Token Index: 924

Text #11

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.8954. Min Act: -0.1700

Data Index: 890272 (Open Web Text)

Max Activating Token Index: 356

Text #12

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.9045. Min Act: -0.1700

Data Index: 6298095 (Open Web Text)

Max Activating Token Index: 266

Text #13

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.9100. Min Act: -0.1700

Data Index: 274780 (Open Web Text)

Max Activating Token Index: 681

Text #14

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.9151. Min Act: -0.1700

Data Index: 8226888 (Open Web Text)

Max Activating Token Index: 552

Text #15

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.9299. Min Act: -0.1700

Data Index: 3017257 (Open Web Text)

Max Activating Token Index: 365

Text #16

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.8976. Min Act: -0.1700

Data Index: 3154218 (Open Web Text)

Max Activating Token Index: 356

Text #17

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.8706. Min Act: -0.1700

Data Index: 4932324 (Open Web Text)

Max Activating Token Index: 999

Text #18

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.8615. Min Act: -0.1700

Data Index: 4951342 (Open Web Text)

Max Activating Token Index: 499

Text #19

Max Range: 4.0055. Min Range: -4.0055

Max Act: 3.8812. Min Act: -0.1700

Data Index: 397098 (Open Web Text)

Max Activating Token Index: 138