< Prev | Home | Model | Random | Next >

Model: SoLU Model: 2 Layers, 2944 Neurons per Layer

Dataset: The Pile

Neuron 2799 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-2l-pile')



Text #0

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0178. Min Act: -0.0003

Data Index: 1667719 (The Pile)

Max Activating Token Index: 186

Click toggle to see full text

Truncated

Full Text #0


Text #1

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0178. Min Act: -0.0002

Data Index: 1737344 (The Pile)

Max Activating Token Index: 904

Click toggle to see full text

Truncated

Full Text #1


Text #2

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0174. Min Act: -0.0003

Data Index: 291172 (The Pile)

Max Activating Token Index: 116

Click toggle to see full text

Truncated

Full Text #2


Text #3

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0169. Min Act: -0.0002

Data Index: 256552 (The Pile)

Max Activating Token Index: 71

Click toggle to see full text

Truncated

Full Text #3


Text #4

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0166. Min Act: -0.0003

Data Index: 1114698 (The Pile)

Max Activating Token Index: 848

Click toggle to see full text

Truncated

Full Text #4


Text #5

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0167. Min Act: -0.0003

Data Index: 1647100 (The Pile)

Max Activating Token Index: 477

Click toggle to see full text

Truncated

Full Text #5


Text #6

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0162. Min Act: -0.0003

Data Index: 37066 (The Pile)

Max Activating Token Index: 879

Click toggle to see full text

Truncated

Full Text #6


Text #7

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0160. Min Act: -0.0003

Data Index: 1981377 (The Pile)

Max Activating Token Index: 986

Click toggle to see full text

Truncated

Full Text #7


Text #8

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0160. Min Act: -0.0003

Data Index: 1735099 (The Pile)

Max Activating Token Index: 794

Click toggle to see full text

Truncated

Full Text #8


Text #9

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0162. Min Act: -0.0003

Data Index: 71018 (The Pile)

Max Activating Token Index: 745

Click toggle to see full text

Truncated

Full Text #9


Text #10

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0162. Min Act: -0.0003

Data Index: 1315590 (The Pile)

Max Activating Token Index: 129

Click toggle to see full text

Truncated

Full Text #10


Text #11

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0158. Min Act: -0.0003

Data Index: 46408 (The Pile)

Max Activating Token Index: 360

Click toggle to see full text

Truncated

Full Text #11


Text #12

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0164. Min Act: -0.0003

Data Index: 1597894 (The Pile)

Max Activating Token Index: 78

Click toggle to see full text

Truncated

Full Text #12


Text #13

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0161. Min Act: -0.0002

Data Index: 1611701 (The Pile)

Max Activating Token Index: 645

Click toggle to see full text

Truncated

Full Text #13


Text #14

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0157. Min Act: -0.0003

Data Index: 381756 (The Pile)

Max Activating Token Index: 588

Click toggle to see full text

Truncated

Full Text #14


Text #15

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0156. Min Act: -0.0003

Data Index: 1042132 (The Pile)

Max Activating Token Index: 509

Click toggle to see full text

Truncated

Full Text #15


Text #16

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0159. Min Act: -0.0003

Data Index: 1136591 (The Pile)

Max Activating Token Index: 236

Click toggle to see full text

Truncated

Full Text #16


Text #17

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0157. Min Act: -0.0003

Data Index: 828553 (The Pile)

Max Activating Token Index: 716

Click toggle to see full text

Truncated

Full Text #17


Text #18

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0154. Min Act: -0.0003

Data Index: 967801 (The Pile)

Max Activating Token Index: 983

Click toggle to see full text

Truncated

Full Text #18


Text #19

Max Range: 0.0178. Min Range: -0.0178

Max Act: 0.0156. Min Act: -0.0003

Data Index: 1532897 (The Pile)

Max Activating Token Index: 899

Click toggle to see full text

Truncated

Full Text #19